Guesswork — LLMs vs a statistical aggregate at predicting benchmark scores
Guesswork is a live benchmark for predicting benchmark scores. Every day new benchmark results land, and this site has already predicted each one. It scores who called it closer: The Aggregate (the statistical model behind the unified rankings) or frontier LLMs given the same information. Predictions are made blind, before the score exists, and graded once it lands.
Latest ranked month: 2026-10 · 61 shared predictions · 34 contestants. Ranked by mean absolute error in per-benchmark z-score units (MAE z, lower is better); “beats Aggregate” is the share of predictions where the contestant landed closer than this site's own.
2026-10 standings
| # | Contestant | MAE (z) | Beats Aggregate |
|---|---|---|---|
| 1 | The Aggregate | 0.523 | — |
| 2 | GPT-6.1 Sol | 0.578 | 43% |
| 3 | Claude Fable 5.1 | 0.581 | 56% |
| 4 | Claude Opus 5.5 | 0.599 | 49% |
| 5 | Claude Sonnet 5.5 | 0.609 | 49% |
| 6 | GPT-6 Astra | 0.613 | 44% |
| 7 | Gemini 3.7 Flash | 0.623 | 46% |
| 8 | Claude Opus 5 | 0.666 | 48% |
| 9 | GPT-5.6 Luna | 0.668 | 46% |
| 10 | GPT-5.6 Terra | 0.679 | 44% |
| 11 | GPT-5.5 | 0.691 | 46% |
| 12 | Claude Opus 4.8 | 0.700 | 51% |
| 13 | Grok 4.6 | 0.716 | 39% |
| 14 | GPT-5.1 | 0.852 | 34% |
| 15 | Mistral Large 3 | 0.879 | 39% |
| 16 | Grok 4.3 | 0.928 | 36% |
| 17 | GLM-5.2 | 0.944 | 36% |
| 18 | Claude Sonnet 5 | 0.951 | 31% |
| 19 | Codestral (2508) | 0.978 | 33% |
| 20 | Qwen 3.7 Max | 0.982 | 36% |
| 21 | Mistral Small 4 | 1.004 | 39% |
| 22 | Ministral 3 3B | 1.028 | 33% |
| 23 | Mistral Small 3 | 1.044 | 30% |
| 24 | Mistral Nemo | 1.050 | 39% |
| 25 | Qwen 3.5 9B | 1.063 | 21% |
Earlier months: 2026-09, 2026-08, 2026-07, 2026-06.
Interactive version: theaggregate.ai/guesswork · How It Works · Data refreshed daily, snapshot 2026-10-15.