CUDAHercules - Module-Level Pass Rate (Tool-Augmented): leaderboard
CUDAHercules (Li et al., 2026) asks models to write CUDA against expert reference implementations, with correctness checked by the validator of each task over several problem sizes or randomized inputs. This board is the share of the 43 architecture-general module-level tasks (a module or kernel family such as attention or FFT) whose code passes, in percent (higher is better), under tool-augmented search, up to 20 rounds with build logs, hardware specifications and GPU profiling tools. Speed relative to the expert code is a separate measure.
Metric: Pass rate (%; share of the 43 architecture-general Class 2 module-level tasks whose generated CUDA passes the task validator; tool-augmented: up to 20 rounds with build logs, hardware specifications and GPU profiling tools (CUDAForge)). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 | 47 | #100 |
| 2 | GPT-5.4 | 2 | #121 |
| 3 | Qwen 3.5 122B A10B | 2 | #329 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=cudahercules-module-level-pass-rate-tool-augmented · How It Works · Data refreshed daily, snapshot 2026-11-12.