ConstraintRot: leaderboard
Whether an LLM agent keeps obeying an in-context governance policy after the harness compacts its conversation into a summary; ConstraintRot (Chen, 2026). Violation rate in percent: the share of 27 episodes (nine tasks, three repetitions) in which the agent makes the tool call the policy forbids after compaction; lower is better.
Metric: Violation rate (%; share of 27 episodes, 9 tasks x 3 repetitions, in which the agent performs the prohibited tool call after its own context compaction dropped an in-context policy it obeyed with full context). Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GLM-5.1 | 0 | #175 |
| 2 | Gemini 3.5 Flash | 4 | #98 |
| 3 | Claude Sonnet 4.6 | 19 | #158 |
| 4 | Qwen 3.6 27B | 30 | #339 |
| 5 | GPT-5.4 Mini | 41 | #389 |
| 6 | Kimi K2.5 | 59 | #252 |
| 7 | DeepSeek V4 Flash | 59 | #254 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=constraintrot · How It Works · Data refreshed daily, snapshot 2026-11-12.