AutoRLBench 37 Issue / 17 Solution - Solution: leaderboard

Solution-implementation split (17 tasks) of Thinking-Space AutoRLBench 37-issue / 17-solution edition measuring whether agents implement working fixes that pass held-out RL training checks; pass rate (%).

Metric: Repaired run passes its rerun, 17 tasks (%). Source: thinking-space.github.io. Saturation forecast: Around November 2028. 9 models tracked. Updated 20 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6.1 SolMax35.3#15
2GPT-6 AstraMax29.4#10
3GLM-5.3Max29.4#73
4Claude Fable 5.1Max29.4#20
5Kimi K3.1 ProMax29.4#35
6Qwen 3.8 MaxMax29.4#66
7Grok 4.7Max29.4#51
8Kimi K3Max17.6#61
9DeepSeek V4 ProMax5.9#198

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-37-issue-17-solution-solution · How It Works · Data refreshed daily, snapshot 2026-11-20.