AutoRLBench 35 Issue / 20 Solution - Issue: leaderboard

AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.

Metric: Root cause found, 35 failed training runs (%). Source: thinking-space.github.io. Saturation forecast: Around November 2027. 10 models tracked. Updated 15 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax42.9#6
2GPT-6.1 SolMax37.1#10
3GLM-5.3Max31.4#68
4Claude Fable 5.1Max25.7#15
5Qwen 3.8 MaxMax25.7#59
6Kimi K3.1 ProMax22.9#29
7Grok 4.7Max22.9#47
8Gemini 4 ArgonMax22.9#12
9Kimi K3Max0#56
10DeepSeek V4 ProMax0#191

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-35-issue-20-solution-issue · How It Works · Data refreshed daily, snapshot 2026-11-16.