ConstraintRot: leaderboard
Metric: Violation rate (%; share of 27 episodes, 9 tasks x 3 repetitions, in which the agent performs the prohibited tool call after its own context compaction dropped an in-context policy it obeyed with full context). Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GLM-5.1 | 0 |
| 2 | Gemini 3.5 Flash | 4 |
| 3 | Claude Sonnet 4.6 | 19 |
| 4 | Qwen 3.6 27B | 30 |
| 5 | GPT-5.4 Mini | 41 |
| 6 | Kimi K2.5 | 59 |
| 7 | DeepSeek V4 Flash | 59 |
Interactive version: theaggregate.ai/benchmark?slug=constraintrot · How It Works · Data refreshed daily, snapshot 2026-09-26.