AutoRLBench 40 Issue / 20 Solution - Issue: leaderboard

Issue-diagnosis split (40 tasks) of Thinking-Space AutoRLBench 40-issue / 20-solution edition measuring whether agents isolate and verify root causes of RL training failures; pass rate (%).

Metric: Root cause found, 40 failed training runs (%). Source: thinking-space.github.io. Saturation forecast: Around July 2028. 9 models tracked. Updated 19 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax30#10
2GPT-6.1 SolMax27.5#15
3GLM-5.3Max20#73
4Claude Fable 5.1Max15#20
5Kimi K3.1 ProMax15#35
6Grok 4.7Max15#51
7Gemini 4 ArgonMax15#18
8Qwen 3.8 MaxMax12.5#66
9Kimi K3Max0#61

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-40-issue-20-solution-issue · How It Works · Data refreshed daily, snapshot 2026-11-20.