AutoRLBench 39 Issue / 20 Solution - Issue: leaderboard

AutoRLBench 39 Issue / 20 Solution - Issue — AutoRLBench native 39-issue/20-solution scored task-set edition, reviewed 16 November 2026. One run per model/task/level; issue compares root-cause answers against a hidden oracle, solution reruns the repaired workspace with metric thresholds and reward-hacking failure. Overall is total passes over59scored runs. README news retains32/24, but the exact shared task-ID sets and every one-decimal table score bind39/20. Native harness and effort are retained. Separate from33/20,32/18 and35/20 editions. Metric: Root cause found, 39 failed training runs (%). Source: https://thinking-space.github.io/AutoRLBench/

Metric: Root cause found, 39 failed training runs (%). Source: thinking-space.github.io. Saturation forecast: Around February 2028. 10 models tracked. Updated 16 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax38.5#6
2GPT-6.1 SolMax33.3#10
3GLM-5.3Max30.8#68
4Claude Fable 5.1Max23.1#15
5Qwen 3.8 MaxMax23.1#59
6Kimi K3.1 ProMax20.5#29
7Grok 4.7Max20.5#47
8Gemini 4 ArgonMax20.5#12
9Kimi K3Max0#56
10DeepSeek V4 ProMax0#191

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-39-issue-20-solution-issue · How It Works · Data refreshed daily, snapshot 2026-11-16.