CheatBench - Software Engineering: leaderboard
The software-engineering category of CheatBench alone: share of 19-20 coding episodes per model, in the vendor's harness, in which the agent cheats (for example by using a reference patch reachable in the repository history). Unlike the headline, it contains no sycophancy component.
Metric: Share of software-engineering episodes (%) in which the agent cheated (e.g. used a reference patch from repository history), provider harness; lower is better. Source: cheatbench.ai. 15 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra | 0 | #6 |
| 2 | Claude Opus 5.5 | 0 | #5 |
| 3 | GPT-6 Sol | 0 | #25 |
| 4 | Gemini 3.8 Flash | 5 | #36 |
| 5 | Gemini 4 Argon | 10 | #11 |
| 6 | Claude Fable 5.1 | 10 | #15 |
| 7 | Grok 4.7 | 15 | #44 |
| 8 | Claude Opus 5 | 25 | #22 |
| 9 | Muse Spark 1.3 | 30 | #46 |
| 10 | Grok 4.6 | 40 | #45 |
| 11 | Muse Wonder | 40 | #28 |
| 12 | Kimi K3 | 52.63 | #54 |
| 13 | DeepSeek V4 Pro | 68.42 | #188 |
| 14 | Muse Spark Open | 75 | #81 |
| 15 | GPT-5.6 Sol | 85 | #30 |
Interactive version: theaggregate.ai/benchmark?slug=cheatbench-software-engineering · How It Works · Data refreshed daily, snapshot 2026-11-13.