AutoRLBench 39 Issue / 20 Solution - Solution: leaderboard

AutoRLBench 39 Issue / 20 Solution - Solution — AutoRLBench native 39-issue/20-solution scored task-set edition, reviewed 16 November 2026. One run per model/task/level; issue compares root-cause answers against a hidden oracle, solution reruns the repaired workspace with metric thresholds and reward-hacking failure. Overall is total passes over59scored runs. README news retains32/24, but the exact shared task-ID sets and every one-decimal table score bind39/20. Native harness and effort are retained. Separate from33/20,32/18 and35/20 editions. Metric: Repaired run passes its rerun, 20 tasks (%). Source: https://thinking-space.github.io/AutoRLBench/

Metric: Repaired run passes its rerun, 20 tasks (%). Source: thinking-space.github.io. Saturation forecast: Around June 2028. 10 models tracked. Updated 16 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 4 ArgonMax60#12
2GPT-6.1 SolMax45#10
3Kimi K3.1 ProMax45#29
4GPT-6 AstraMax40#6
5GLM-5.3Max40#68
6Claude Fable 5.1Max40#15
7Qwen 3.8 MaxMax30#59
8Grok 4.7Max25#47
9Kimi K3Max15#56
10DeepSeek V4 ProMax15#191

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-39-issue-20-solution-solution · How It Works · Data refreshed daily, snapshot 2026-11-16.