AutoRLBench 37 Issue / 17 Solution: leaderboard

Thinking-Space AutoRLBench 37-issue / 17-solution edition (commit 5cfcc52d) evaluating autonomous RL research agents across 54 tasks (37 issue diagnosis and 17 solution implementation tasks) in sandboxed repositories; overall pass rate (%).

Metric: Pass rate over 37 issue judgments and 17 solution runs (%). Source: thinking-space.github.io. Saturation forecast: Around 2029. 8 models tracked. Updated 20 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6.1 SolMax33.3#15
2GPT-6 AstraMax31.5#10
3GLM-5.3Max31.5#73
4Grok 4.7Max25.9#51
5Claude Fable 5.1Max24.1#20
6Kimi K3.1 ProMax20.4#35
7Qwen 3.8 MaxMax20.4#66
8Kimi K3Max9.3#61

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-37-issue-17-solution · How It Works · Data refreshed daily, snapshot 2026-11-20.