AutoRLBench 35 Issue / 20 Solution - Solution: leaderboard

AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.

Metric: Repaired run passes its rerun, 20 tasks (%). Source: thinking-space.github.io. Saturation forecast: Around June 2028. 10 models tracked. Updated 15 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 4 ArgonMax60#12
2GPT-6.1 SolMax45#10
3Kimi K3.1 ProMax45#29
4GPT-6 AstraMax40#6
5GLM-5.3Max40#68
6Claude Fable 5.1Max40#15
7Qwen 3.8 MaxMax30#59
8Grok 4.7Max25#47
9Kimi K3Max15#56
10DeepSeek V4 ProMax15#191

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-35-issue-20-solution-solution · How It Works · Data refreshed daily, snapshot 2026-11-16.