DistDebug-Bench - Block precision: leaderboard
DistDebug Table 9:120 common incidents,three selected clean attempts per bug,mean within bug then equal across bugs; native file/function/line/causal-block/mention localization and block precision. Flagged answer-leaking attempts discarded and rerun; no best-attempt selection. Explicit medium effort for GPT and Claude; source says medium where controls exist and provider defaults otherwise. Model/harness configurations stay separate. These are root-cause localization metrics, not patch success. Metric: Precision (%).
Metric: Precision (%). Source: arxiv.org. Saturation forecast: Around 2034. 17 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.5 Flash | 51.1 | #66 |
| 2 | Kimi K2.6 | 45.9 | #113 |
| 3 | Claude Opus 5.5 (Medium) | 45.5 | #3 (Claude Opus 5.5) |
| 4 | GPT-6 Astra (Medium) | 42.7 | #6 (GPT-6 Astra) |
| 5 | GPT-5.3 Codex (Medium) | 40 | #72 (GPT-5.3 Codex) |
| 6 | GPT-5.4 (Medium) | 39.5 | #83 (GPT-5.4) |
| 7 | Hy3 | 38.7 | #100 |
| 8 | MiMo-V2.5-Pro | 38.1 | #167 |
| 9 | MiMo-V2.5 | 36.5 | #196 |
| 10 | GPT-5.5 (Medium) | 34.7 | #33 (GPT-5.5) |
| 11 | O4 Mini (Medium) | 34.7 | #185 (O4 Mini) |
| 12 | DeepSeek V4 Flash | 32.7 | #155 |
No result here: #7 GPT-6.1 Sol, #9 Gemini 4 Argon, #10 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=distdebug-bench-block-precision · How It Works · Data refreshed daily, snapshot 2026-11-05.