CyberGym-E2E-AA: leaderboard
Artificial Analysis's end-to-end CyberGym run: 131 memory-safety vulnerabilities from 131 C/C++ projects, one 90-minute attempt each on AA's Stirrup harness; a task scores only if the agent's proof of concept crashes the unpatched build, its source patch stops the crash and the project's tests still pass. Cells for models that declined more than 5% of tasks are withheld. Pass@1 (%).
Metric: Pass@1 (%). Source: artificialanalysis.ai. Saturation forecast: Not forecast. 17 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Kimi K3.1 Pro | 94.66 | #17 |
| 2 | Mistral Large 4 | 81.68 | #106 |
| 3 | mistral-large-4-oss (High) | 79.39 | #60 (mistral-large-4-oss) |
| 4 | MiMo-V2.6-Pro | 78.63 | #48 |
| 5 | GPT-6 Luna (Max) | 77.86 | #61 (GPT-6 Luna) |
| 6 | DeepSeek V4.1 Flash (1017) (Max) | 76.34 | #27 (DeepSeek V4.1 Flash (1017)) |
| 7 | Step 5 Preview | 74.81 | #53 |
| 8 | GLM-5.3 Flash | 74.05 | #77 |
| 9 | Grok 4.7 (xHigh) | 74.05 | #26 (Grok 4.7) |
| 10 | Muse Spark 1.3 (xHigh) | 59.54 | #31 (Muse Spark 1.3) |
| 11 | Kimi K3 (Max) | 58.02 | #35 (Kimi K3) |
| 12 | Beam 1.0 | 45.04 | #102 |
| 13 | GLM-5.3 (Max) | 29.01 | #44 (GLM-5.3) |
| 14 | MiniMax-M3 | 24.43 | #151 |
| 15 | DeepSeek V4.1 Flash (Max) | 22.9 | #37 (DeepSeek V4.1 Flash) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=cybergym-e2e-aa · How It Works · Data refreshed daily, snapshot 2026-11-05.