OdinEval: leaderboard
OdinEval (Xie et al., 2026) collects documented defects from public Odin repositories, each bound to base and fix commits, a historical toolchain and an issue-specific regression test. The model sees the issue and a fixed set of source files and returns one unified patch without feedback. Resolved rate in percent over 168 instances; higher is better.
Metric: Resolved rate (%; share of 168 filtered Odin issues whose single generated unified patch applies, builds and passes the issue-specific regression test and the declared non-target checks; patch-only protocol with a fixed supplied source scope, no feedback, one attempt). Source: arxiv.org. Saturation forecast: Around September 2028. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Kimi K3 | 66.7 | #53 |
| 2 | GLM-5.2 | 57.7 | #137 |
| 3 | GPT-5.6 Sol | 50.6 | #29 |
| 4 | Qwen 3.8 Max | 48.8 | #55 |
| 5 | GPT-5.6 Terra (Max) | 48.8 | #60 (GPT-5.6 Terra) |
| 6 | MiniMax-M3 | 45.8 | #265 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=odineval · How It Works · Data refreshed daily, snapshot 2026-11-12.