AutoRLBench 39 Issue / 20 Solution: leaderboard

AutoRLBench 39 Issue / 20 Solution — AutoRLBench native 39-issue/20-solution scored task-set edition, reviewed 16 November 2026. One run per model/task/level; issue compares root-cause answers against a hidden oracle, solution reruns the repaired workspace with metric thresholds and reward-hacking failure. Overall is total passes over59scored runs. README news retains32/24, but the exact shared task-ID sets and every one-decimal table score bind39/20. Native harness and effort are retained. Separate from33/20,32/18 and35/20 editions. Metric: Pass rate over 39 issue judgments and 20 solution runs (%). Source: https://thinking-space.github.io/AutoRLBench/

Metric: Pass rate over 39 issue judgments and 20 solution runs (%). Source: thinking-space.github.io. Saturation forecast: Around April 2028. 10 models tracked. Updated 16 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax39#6
2GPT-6.1 SolMax37.3#10
3GLM-5.3Max33.9#68
4Gemini 4 ArgonMax33.9#12
5Claude Fable 5.1Max28.8#15
6Kimi K3.1 ProMax28.8#29
7Qwen 3.8 MaxMax25.4#59
8Grok 4.7Max22#47
9Kimi K3Max5.1#56
10DeepSeek V4 ProMax5.1#191

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-39-issue-20-solution · How It Works · Data refreshed daily, snapshot 2026-11-16.