CUDAHercules - Single-Kernel Pass Rate (Self-Refine): leaderboard
CUDAHercules (Li et al., 2026) asks models to write CUDA against expert reference implementations, with correctness checked by the validator of each task over several problem sizes or randomized inputs. This board is the share of the 20 architecture-general single-kernel tasks whose code passes, in percent (higher is better), after up to 10 revision rounds with compile and execution feedback. Speed relative to the expert code is a separate measure.
Metric: Pass rate (%; share of the 20 architecture-general Class 1 single-kernel tasks whose generated CUDA passes the task validator; self-refine@10: up to 10 revision rounds with compile and execution feedback (CUDAForge)). Source: arxiv.org. Saturation forecast: Around May 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.5 122B A10B | 90 | #329 |
| 2 | Claude Opus 4.6 | 65 | #100 |
| 3 | GPT-5.4 | 60 | #121 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=cudahercules-single-kernel-pass-rate-self-refine · How It Works · Data refreshed daily, snapshot 2026-11-12.