Guesswork — LLMs vs a statistical aggregate at predicting benchmark scores
Guesswork is a live benchmark for predicting benchmark scores. Every day new benchmark results land, and this site has already predicted each one. It scores who called it closer: The Aggregate (the statistical model behind the unified rankings) or frontier LLMs given the same information. Predictions are made blind, before the score exists, and graded once it lands.
Current month 2026-11 is in progress: 20 of 50 shared predictions so far. Standings are published at 50.
Latest ranked month: 2026-10 · 129 shared predictions · 34 contestants. Ranked by mean absolute error in per-benchmark z-score units (MAE z, lower is better); “beats Aggregate” is the share of predictions where the contestant landed closer than this site's own.
2026-10 standings
| # | Contestant | MAE (z) | Beats Aggregate |
|---|---|---|---|
| 1 | The Aggregate | 0.755 | — |
| 2 | Claude Opus 5.5 | 0.764 | 51% |
| 3 | GPT-6.1 Sol | 0.783 | 45% |
| 4 | GPT-6 Astra | 0.803 | 45% |
| 5 | Claude Sonnet 5.5 | 0.804 | 48% |
| 6 | GPT-5.6 Luna | 0.842 | 45% |
| 7 | Claude Fable 5.1 | 0.858 | 49% |
| 8 | Grok 4.6 | 0.859 | 40% |
| 9 | Claude Opus 4.8 | 0.860 | 47% |
| 10 | Gemini 3.7 Flash | 0.867 | 44% |
| 11 | Claude Opus 5 | 0.872 | 46% |
| 12 | GPT-5.6 Terra | 0.914 | 41% |
| 13 | GPT-5.5 | 0.936 | 43% |
| 14 | GPT-5.1 | 0.987 | 40% |
| 15 | Mistral Large 3 | 1.011 | 39% |
| 16 | Claude Sonnet 5 | 1.023 | 34% |
| 17 | Grok 4.3 | 1.054 | 35% |
| 18 | Ministral 3 3B | 1.059 | 38% |
| 19 | GLM-5.2 | 1.104 | 35% |
| 20 | Mistral Small 4 | 1.119 | 35% |
| 21 | Qwen 3.7 Max | 1.121 | 38% |
| 22 | Codestral (2508) | 1.138 | 35% |
| 23 | GPT-OSS 20B | 1.163 | 33% |
| 24 | Mistral Small 3 | 1.171 | 31% |
| 25 | Ministral 3 14B | 1.180 | 31% |
Earlier months: 2026-09, 2026-08, 2026-07, 2026-06.
Interactive version: theaggregate.ai/guesswork · How It Works · Data refreshed daily, snapshot 2026-11-05.