AutoRLBench 37 Issue / 17 Solution - Issue: leaderboard
Issue-diagnosis split (37 tasks) of Thinking-Space AutoRLBench 37-issue / 17-solution edition measuring whether agents isolate and verify root causes of RL training failures; pass rate (%).
Metric: Root cause found, 37 failed training runs (%). Source: thinking-space.github.io. Saturation forecast: Around 2029. 9 models tracked. Updated 20 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | DeepSeek V4.1 Flash | Max | 35.1 | #69 |
| 2 | GPT-6 Astra | Max | 32.4 | #10 |
| 3 | GPT-6.1 Sol | Max | 32.4 | #15 |
| 4 | GLM-5.3 | Max | 32.4 | #73 |
| 5 | Grok 4.7 | Max | 24.3 | #51 |
| 6 | Claude Fable 5.1 | Max | 21.6 | #20 |
| 7 | Kimi K3.1 Pro | Max | 16.2 | #35 |
| 8 | Qwen 3.8 Max | Max | 16.2 | #66 |
| 9 | Kimi K3 | Max | 5.4 | #61 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=autorlbench-37-issue-17-solution-issue · How It Works · Data refreshed daily, snapshot 2026-11-20.