AutoRLBench 36 Issue / 18 Solution - Issue: leaderboard

AutoRLBench native36-issue/18-solution scored task-set edition, reviewed 17 November2026. Exact shared task IDs and every printed rate are checked against one run per model/task/level. Issue judgments use the hidden oracle; solution reruns use metric thresholds with reward-hacking failure. Overall is total passes over54scored runs. This is a separate edition from39/20, 35/20, 33/20 and32/18; historical observations retain their original cohort. Metric: Root cause found, 36 failed training runs (%). Source: https://thinking-space.github.io/AutoRLBench/

Metric: Root cause found, 36 failed training runs (%). Source: thinking-space.github.io. Saturation forecast: Around March 2028. 10 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax33.3#8
2GPT-6.1 SolMax30.6#12
3GLM-5.3Max22.2#73
4Claude Fable 5.1Max16.7#17
5Kimi K3.1 ProMax16.7#32
6Grok 4.7Max16.7#48
7Gemini 4 ArgonMax16.7#14
8Qwen 3.8 MaxMax13.9#64
9Kimi K3Max0#59
10DeepSeek V4 ProMax0#198

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=autorlbench-36-issue-18-solution-issue · How It Works · Data refreshed daily, snapshot 2026-11-17.