AutoRLBench 36 Issue / 18 Solution - Solution: leaderboard

AutoRLBench native36-issue/18-solution scored task-set edition, reviewed 17 November2026. Exact shared task IDs and every printed rate are checked against one run per model/task/level. Issue judgments use the hidden oracle; solution reruns use metric thresholds with reward-hacking failure. Overall is total passes over54scored runs. This is a separate edition from39/20, 35/20, 33/20 and32/18; historical observations retain their original cohort. Metric: Repaired run passes its rerun, 18 tasks (%). Source: https://thinking-space.github.io/AutoRLBench/

Metric: Repaired run passes its rerun, 18 tasks (%). Source: thinking-space.github.io. Saturation forecast: Around June 2028. 10 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 4 ArgonMax55.6#14
2GPT-6 AstraMax38.9#8
3GPT-6.1 SolMax38.9#12
4Kimi K3.1 ProMax38.9#32
5GLM-5.3Max33.3#73
6Claude Fable 5.1Max27.8#17
7Qwen 3.8 MaxMax27.8#64
8Kimi K3Max16.7#59
9Grok 4.7Max16.7#48
10DeepSeek V4 ProMax5.6#198

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-36-issue-18-solution-solution · How It Works · Data refreshed daily, snapshot 2026-11-17.