AtomicCommitBench: leaderboard
AtomicCommitBench (2026) gives a coding agent the squashed diff of 2 to 8 consecutive commits by one author, 800 episodes from 10 Python projects, and asks it to group the hunks into a replayable sequence of atomic commits. Score is the adjusted Rand index between the predicted grouping and the developer history, averaged over episodes, from -1 to 1 with 0 at chance; higher is better.
Metric: Adjusted Rand index against the observed developer commit history (-1 to 1, 0 for chance agreement; mean over 800 squashed multi-commit episodes from 10 Python projects, grouping the hunks of each squashed diff into replayable commits; one run per episode, at most 50 turns). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 0.46 | #123 |
| 2 | GLM-5 | 0.43 | #260 |
| 3 | MiniMax-M2.5 | 0.31 | #582 |
| 4 | Kimi K2.5 | 0.29 | #256 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=atomiccommitbench · How It Works · Data refreshed daily, snapshot 2026-11-13.