AutoRLBench 36 Issue / 18 Solution: leaderboard

AutoRLBench native36-issue/18-solution scored task-set edition, reviewed 17 November2026. Exact shared task IDs and every printed rate are checked against one run per model/task/level. Issue judgments use the hidden oracle; solution reruns use metric thresholds with reward-hacking failure. Overall is total passes over54scored runs. This is a separate edition from39/20, 35/20, 33/20 and32/18; historical observations retain their original cohort. Metric: Pass rate over 36 issue judgments and 18 solution runs (%). Source: https://thinking-space.github.io/AutoRLBench/

Metric: Pass rate over 36 issue judgments and 18 solution runs (%). Source: thinking-space.github.io. Saturation forecast: Around May 2028. 10 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax35.2#8
2GPT-6.1 SolMax33.3#12
3Gemini 4 ArgonMax29.6#14
4GLM-5.3Max25.9#73
5Kimi K3.1 ProMax24.1#32
6Claude Fable 5.1Max20.4#17
7Qwen 3.8 MaxMax18.5#64
8Grok 4.7Max16.7#48
9Kimi K3Max5.6#59
10DeepSeek V4 ProMax1.9#198

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-36-issue-18-solution · How It Works · Data refreshed daily, snapshot 2026-11-17.