AutoRLBench 36 Issue / 18 Solution - Solution: leaderboard
AutoRLBench native36-issue/18-solution scored task-set edition, reviewed 17 November2026. Exact shared task IDs and every printed rate are checked against one run per model/task/level. Issue judgments use the hidden oracle; solution reruns use metric thresholds with reward-hacking failure. Overall is total passes over54scored runs. This is a separate edition from39/20, 35/20, 33/20 and32/18; historical observations retain their original cohort. Metric: Repaired run passes its rerun, 18 tasks (%). Source: https://thinking-space.github.io/AutoRLBench/
Metric: Repaired run passes its rerun, 18 tasks (%). Source: thinking-space.github.io. Saturation forecast: Around June 2028. 10 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 4 Argon | Max | 55.6 | #14 |
| 2 | GPT-6 Astra | Max | 38.9 | #8 |
| 3 | GPT-6.1 Sol | Max | 38.9 | #12 |
| 4 | Kimi K3.1 Pro | Max | 38.9 | #32 |
| 5 | GLM-5.3 | Max | 33.3 | #73 |
| 6 | Claude Fable 5.1 | Max | 27.8 | #17 |
| 7 | Qwen 3.8 Max | Max | 27.8 | #64 |
| 8 | Kimi K3 | Max | 16.7 | #59 |
| 9 | Grok 4.7 | Max | 16.7 | #48 |
| 10 | DeepSeek V4 Pro | Max | 5.6 | #198 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=autorlbench-36-issue-18-solution-solution · How It Works · Data refreshed daily, snapshot 2026-11-17.