AutoRLBench 45 Issue / 19 Solution - Solution: leaderboard

Solution-implementation split (19 tasks) of Thinking-Space AutoRLBench 45-issue / 19-solution edition measuring whether agents implement working fixes that pass held-out RL training checks; pass rate (%).

Metric: Repaired run passes its rerun, 19 tasks (%). Source: thinking-space.github.io. Saturation forecast: Around 2029. 10 models tracked. Updated 21 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6.1 SolMax31.6#15
2Qwen 3.8 MaxMax31.6#63
3GPT-6 AstraMax26.3#10
4DeepSeek V4.1 FlashMax26.3#67
5GLM-5.3Max26.3#72
6Claude Fable 5.1Max26.3#20
7Kimi K3.1 ProMax26.3#36
8Grok 4.7Max26.3#51
9Kimi K3Max15.8#61
10DeepSeek V4 ProMax5.3#196

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-45-issue-19-solution-solution · How It Works · Data refreshed daily, snapshot 2026-11-21.