Puzzle portfolio ladder

Our Benchmarks

30 generated puzzles in five families: logic grids, knapsacks, document facts, specifications and bug hunts.

Run Aug 29 – Nov 13, 2026, 30 items.

Method

Six items from each of five generators whose answers are fixed by construction: zebra logic grids, knapsack optimisation, fact lookup in a long document, checking an implementation against its specification, and finding a planted bug. Difficulty rises within each family, and the bank is frozen.

Run at each provider's default reasoning effort, like the ladder above.

Results

Puzzle portfolio ladder: Accuracy, highest first
#ModelAccuracyItems
1GPT-5.296.7%30
2GPT-5.596.7%30
3MiniMax-M3.1-Flash (free route)96.7%30
4Claude Fable 5.296.7%30
5Particle-1 (free route)96.7%30
6Ling 3.1 Flash96.3%27
7GLM-5.3 (Mistral API)92.9%28
8Nemotron 3.5 Super VL (free route)92.6%27
9Kimi K2.690.0%30
10Claude Opus 4.890.0%30
11Qwen 3.6 Plus86.7%30
12DeepSeek V4 Pro86.7%30
13ERNIE 5.5 Flash Preview (free route)86.2%29
14GPT-OSS-120B83.3%30
15Nex N2.5 Pro (free route)82.1%28
16Nemotron 3 Ultra (free route)77.8%27
17DeepSeek V4.1 Flash (1017) (Mistral API)76.7%30
18Gemma 4 31B (IT) (free route)73.3%30
19Nemotron 3 Super (free route)66.7%27
20Flint (free route)66.7%30
21Ling-3.0-flash-VL (free route)63.3%30
22GPT-5.460.0%30
23Space Bunny Alpha (stealth release)60.0%30
24Gemma 4 26B A4B (IT) (free route)53.3%30
25Laguna XS 2.1 (free route)44.8%29
26Kimi K2 090543.3%30
  • GPT-5.4 solves 29 of 30 when it is asked for high reasoning effort.

In the main leaderboard

These results are collected for the main leaderboard alongside external benchmarks. New scores enter the rating through the regular audited data updates. Comparisons with that rating therefore share some of its inputs.

How the rating works

Interactive version: theaggregate.ai/our-benchmarks/puzzle-portfolio · How It Works · Data refreshed daily, snapshot 2026-11-14.