AutoRLBench 35 Issue / 20 Solution - Solution: leaderboard
AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.
Metric: Repaired run passes its rerun, 20 tasks (%). Source: thinking-space.github.io. Saturation forecast: Around June 2028. 10 models tracked. Updated 15 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 4 Argon | Max | 60 | #12 |
| 2 | GPT-6.1 Sol | Max | 45 | #10 |
| 3 | Kimi K3.1 Pro | Max | 45 | #29 |
| 4 | GPT-6 Astra | Max | 40 | #6 |
| 5 | GLM-5.3 | Max | 40 | #68 |
| 6 | Claude Fable 5.1 | Max | 40 | #15 |
| 7 | Qwen 3.8 Max | Max | 30 | #59 |
| 8 | Grok 4.7 | Max | 25 | #47 |
| 9 | Kimi K3 | Max | 15 | #56 |
| 10 | DeepSeek V4 Pro | Max | 15 | #191 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=autorlbench-35-issue-20-solution-solution · How It Works · Data refreshed daily, snapshot 2026-11-16.