OctoBench - Claude Code - Checkitem Success: leaderboard
Repository, skill and tool-schema instruction following during coding. Paper v2 Table 3: each scaffold averages the same three judges and uses scaffold-default inference settings. ISR requires all check items in an instance; CSR scores individual check items. The differing HF headline snapshot is not combined. This series reports Checkitem Success Rate (%). Source: https://arxiv.org/html/2601.10343v2.
Metric: Checkitem Success Rate (%). Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.5 | 84.39 | #87 |
| 2 | MiniMax-M2.1 | 82.75 | #299 |
| 3 | Gemini 3 Pro | 80.88 | #82 |
| 4 | Claude Sonnet 4.5 | 80.85 | #148 |
| 5 | GLM-4.6 | 80.07 | #265 |
| 6 | MiniMax-M2 | 79.88 | #327 |
| 7 | Kimi K2 (Thinking) | 79.61 | #252 (Kimi K2) |
| 8 | Seed 1.8 | 79.19 | #143 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=octobench-claude-code-checkitem-success · How It Works · Data refreshed daily, snapshot 2026-11-05.