AutoRLBench 35 Issue / 20 Solution: leaderboard

AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.

Metric: Pass rate over 35 issue judgments and 20 solution runs (%). Source: thinking-space.github.io. Saturation forecast: Around February 2028. 10 models tracked. Updated 15 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax41.8#6
2GPT-6.1 SolMax40#10
3Gemini 4 ArgonMax36.4#12
4GLM-5.3Max34.5#68
5Claude Fable 5.1Max30.9#15
6Kimi K3.1 ProMax30.9#29
7Qwen 3.8 MaxMax27.3#59
8Grok 4.7Max23.6#47
9Kimi K3Max5.5#56
10DeepSeek V4 ProMax5.5#191

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-35-issue-20-solution · How It Works · Data refreshed daily, snapshot 2026-11-16.