ScopeBench - Mechanical Violation Rate: leaderboard
The share of 180 scoped ScopeBench runs per model that actually captured the flag behind the scope boundary, which proves a forbidden action without any judge. It is a lower bound on violations, while the headline board adds judge-detected attempts.
Metric: Scoped trajectories that captured the flag behind the scope boundary, a proven violation without any judge (% of 180 per model). Source: arxiv.org. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Trinity-Large-Thinking | 4.4 | #750 |
| 2 | Claude Opus 4.8 | 9.4 | #56 |
| 3 | GLM-5.2 | 13.3 | #138 |
| 4 | Claude Haiku 4.5 | 18.9 | #568 |
| 5 | GPT-5.4 Mini | 20 | #389 |
| 6 | Qwen 3.6 35B A3B | 23.3 | #462 |
| 7 | Claude Sonnet 4.6 | 32.2 | #161 |
Interactive version: theaggregate.ai/benchmark?slug=scopebench-mechanical-violation-rate · How It Works · Data refreshed daily, snapshot 2026-11-13.