Inversion and execution ladder
24 generated puzzles: trace a short program, or undo a hidden digit transform, at rising difficulty.
Run Aug 28 – Oct 3, 2026, 24 items.
Method
Half the items give the model a short generated program and ask for its output; the other half give the result of a hidden digit transform and ask for the input that produced it. Six rungs of difficulty and exact-match answers, on items generated for this ladder from a fixed seed.
Run at each provider's default reasoning effort, which this ladder is very sensitive to. The scores show what a model does with its out-of-the-box settings, not its ceiling.
Results
| # | Model | Accuracy | Items |
|---|---|---|---|
| 1 | GPT-5.5 | 100.0% | 24 |
| 2 | Dots3-Note (Preview) (free route) | 100.0% | 23 |
| 3 | GPT-5.2 | 91.7% | 24 |
| 4 | GLM-5.3 (Mistral API) | 91.7% | 24 |
| 5 | Ling 3.1 Flash | 91.7% | 24 |
| 6 | Kimi K2.6 | 87.5% | 24 |
| 7 | Claude Opus 4.8 | 83.3% | 24 |
| 8 | Nemotron 3 Super (free route) | 83.3% | 24 |
| 9 | GPT-OSS-120B | 79.2% | 24 |
| 10 | Ling-3.0-flash-VL (free route) | 78.3% | 23 |
| 11 | Nex N2.5 Pro (free route) | 70.8% | 24 |
| 12 | Nex N2.5 Mini (free route) | 50.0% | 22 |
| 13 | GPT-5.4 | 41.7% | 24 |
| 14 | Gemma 4 26B A4B (IT) (free route) | 41.7% | 24 |
| 15 | Space Bunny Alpha (stealth release) | 26.1% | 23 |
| 16 | Gemma 4 31B (IT) (free route) | 25.0% | 24 |
| 17 | Kimi K2 0905 | 20.8% | 24 |
- GPT-5.4 solves 23 of 24 when it is asked for high reasoning effort.
- Not ranked, too few answered items: Ling-3.0-flash-fin (21 of 24), Ling-3.0-flash-sante (21 of 24), GLM-5.2 (21 of 24), Qwen 3.6 Plus (20 of 24), North Mini Code (19 of 24), Nemotron 3 Ultra (19 of 24), Nemotron 3 Nano Omni 30B-A3B (Reasoning) (19 of 24), Laguna XS 2.1 (18 of 24), Apodex 1.1 Mini (18 of 24), DeepSeek V4 Pro (15 of 24), Laguna S 2.1 (15 of 24), Nemotron 3.5 Lightning (13 of 24), Qwen 3.8 27B (9 of 24), LFM2.5-2.6B (3 of 24), Inkling Small (0 of 24), MiniMax-M3 (0 of 24), MiniMax-M2.7 (0 of 24).
In the main leaderboard
These results are collected for the main leaderboard alongside external benchmarks. New scores enter the rating through the regular audited data updates. Comparisons with that rating therefore share some of its inputs.
Interactive version: theaggregate.ai/our-benchmarks/inversion-ladder · How It Works · Data refreshed daily, snapshot 2026-10-11.