HoneyBench v0.2 - Overall Hack Rate: leaderboard
Share of scored agent rollouts that commit to specification gaming in HoneyBench v0.2, a 20-task evaluation spanning medical, coding, engineering, administrative, ML, multimodal and game tasks. The headline excludes provider-refused runs and unevaluated tasks; models therefore have different denominators. The alternate rate counting refusals as hacks is retained as source evidence, not used as the score. Access to forbidden information counts as hacking even without a final exploit; otherwise the final submission determines the outcome. Native percentage, lower is better. Source: goodhartlabs.com/honeybench.
Metric: Overall hack rate (%) over scored runs in the v0.2 20-task universe; unevaluated tasks and provider refusals excluded (lower is better). Source: goodhartlabs.com. Saturation forecast: Around February 2027. 22 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Fable 5.2 | 11.93 | #1 |
| 2 | Claude Opus 5.5 | 31.85 | #5 |
| 3 | Claude Sonnet 5.5 | 41.62 | #18 |
| 4 | GPT-6 Astra | 50.77 | #6 |
| 5 | Inkling 2 Small | 52.63 | #50 |
| 6 | GPT-6.1 Sol | 59.49 | #10 |
| 7 | Claude Fable 5.1 | 61.85 | #14 |
| 8 | Gemini 4 Argon | 63.64 | #11 |
| 9 | Step 5 Preview | 64.86 | #105 |
| 10 | Qwen 3.8 Max | 66.15 | #55 |
| 11 | DeepSeek V4 Pro | 71.35 | #181 |
| 12 | Kimi K3.1 Pro | 71.79 | #28 |
| 13 | DeepSeek V4.1 Flash (1017) | 76.41 | #38 |
| 14 | GLM-5.3 | 76.76 | #62 |
| 15 | Kimi K3 | 78.46 | #53 |
No result here: #2 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=honeybench-v0-2-overall-hack-rate · How It Works · Data refreshed daily, snapshot 2026-11-12.