AutoRLBench 37 Issue / 17 Solution - Issue: leaderboard

Issue-diagnosis split (37 tasks) of Thinking-Space AutoRLBench 37-issue / 17-solution edition measuring whether agents isolate and verify root causes of RL training failures; pass rate (%).

Metric: Root cause found, 37 failed training runs (%). Source: thinking-space.github.io. Saturation forecast: Around 2029. 9 models tracked. Updated 20 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1DeepSeek V4.1 FlashMax35.1#69
2GPT-6 AstraMax32.4#10
3GPT-6.1 SolMax32.4#15
4GLM-5.3Max32.4#73
5Grok 4.7Max24.3#51
6Claude Fable 5.1Max21.6#20
7Kimi K3.1 ProMax16.2#35
8Qwen 3.8 MaxMax16.2#66
9Kimi K3Max5.4#61

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-37-issue-17-solution-issue · How It Works · Data refreshed daily, snapshot 2026-11-20.