AutoRLBench 40 Issue / 20 Solution - Solution: leaderboard
Solution-implementation split (20 tasks) of Thinking-Space AutoRLBench 40-issue / 20-solution edition measuring whether agents implement working fixes that pass held-out RL training checks; pass rate (%).
Metric: Repaired run passes its rerun, 20 tasks (%). Source: thinking-space.github.io. Saturation forecast: Around September 2028. 7 models tracked. Updated 19 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-6.1 Sol | Max | 40 | #15 |
| 2 | GLM-5.3 | Max | 35 | #73 |
| 3 | Qwen 3.8 Max | Max | 30 | #66 |
| 4 | Claude Fable 5.1 | Max | 25 | #20 |
| 5 | Kimi K3 | Max | 20 | #61 |
| 6 | Grok 4.7 | Max | 20 | #51 |
| 7 | DeepSeek V4 Pro | Max | 10 | #198 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=autorlbench-40-issue-20-solution-solution · How It Works · Data refreshed daily, snapshot 2026-11-20.