The Aggregate - Trip Outcome Prediction (5 Questions): leaderboard
The Aggregate runs this evaluation itself. Two travellers wrote five questions about what actually happened on a two-day trip to the Baie de Somme, France, in October 2026, and answered them before any model saw the questions. Each model sees only the anonymized trip plan, including the packing list, and one question at a time, and gives a single best guess: what surprised them in town at 18:00, what disappointed them on the guided bay walk, which unpacked item they most wished for, what was inconvenient after the walk and how they solved it, and what the two taxi rides cost in total. A judge model (GPT-6.1 Sol) grades the four open answers against the travellers’ own answers with fixed rubrics (1, ½ or 0); the taxi total earns one minus its relative error, floored at 0. Two runs share the frozen questions and judge: a first panel at medium reasoning with a 4,096-token output cap, and a panel of current leaderboard models at their highest reasoning effort with a 32,768-token cap; each row is labelled with its effort. No tools are used. The score is the mean credit as a percentage; with five questions, close scores do not establish an ordering. Only models answering all five questions are ranked.
Metric: Accuracy (%). Source: theaggregate.ai. Source: theaggregate.ai. Saturation forecast: Around 2030. 30 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 (Max) | 69.8 | #3 (Claude Opus 5.5) |
| 2 | GPT-6.1 Sol (Max) | 69.8 | #7 (GPT-6.1 Sol) |
| 3 | GPT-5.6 Sol (Medium) | 69.8 | #22 (GPT-5.6 Sol) |
| 4 | GPT-6 Astra (Max) | 59.8 | #6 (GPT-6 Astra) |
| 5 | Claude Fable 5.1 (Max) | 59.8 | #10 (Claude Fable 5.1) |
| 6 | GPT-6.1 Sol (Medium) | 59.8 | #7 (GPT-6.1 Sol) |
| 7 | Muse Wonder (Max) | 58.7 | #18 (Muse Wonder) |
| 8 | GPT-6 Astra (Medium) | 58.7 | #6 (GPT-6 Astra) |
| 9 | Gemini 3.8 Flash (Medium) | 58.7 | #30 (Gemini 3.8 Flash) |
| 10 | Claude Opus 5.5 (Medium) | 58.7 | #3 (Claude Opus 5.5) |
| 11 | GPT-6.1 Terra (Max) | 58.4 | #15 (GPT-6.1 Terra) |
| 12 | Claude Sonnet 5.5 (Max) | 57.3 | #14 (Claude Sonnet 5.5) |
| 13 | Gemini 3.7 Flash (Medium) | 57.3 | #24 (Gemini 3.7 Flash) |
| 14 | Gemini 3.6 Flash (Medium) | 57.3 | #57 (Gemini 3.6 Flash) |
| 15 | DeepSeek V4.1 Flash (1017) (Max) | 55.9 | #27 (DeepSeek V4.1 Flash (1017)) |
Method and evidence
Run by The Aggregate · Nov 2, 2026
Read method and notes
Two travellers followed a plan for a weekend in the Baie de Somme, France, at the end of October 2026: trains, two taxis, a bird park, a night in Le Crotoy, a guided walk on the bay and a heritage train home. After the trip they wrote five questions about what actually happened and answered them before any model saw the questions.
Each model sees only the anonymized plan, including the packing list, and one question at a time, and gives a single best guess: what surprised them in town at 18:00, what disappointed them on the walk, which missing item they wished they had packed, what was inconvenient after the walk and how they solved it, and what the two taxi rides cost.
One call per model and question and no tools. The first run requested medium reasoning with a 4,096-token output cap including reasoning; the second asked each model for its highest supported effort (max, or xhigh or high where that is the top setting) with a 32,768-token cap. Provider meanings of each setting can differ. Only models answering all five questions are ranked.
- Five questions, one answer per model and question. Q1-Q4 earn 1, ½ or 0 from a judge model (GPT-6.1 Sol, low reasoning) applying fixed rubrics to the travellers' own answers; the taxi total earns 1 minus its relative error, floored at 0. The score is the mean credit as a percentage.
- Two runs share the frozen plan, questions, key and judge: a first panel at medium reasoning and a panel of current leaderboard models at their highest reasoning effort. Each row is labelled with its effort.
- One trip and five questions: a single different answer moves a score by up to 20 points, so close scores do not establish an ordering.
Interactive version: theaggregate.ai/benchmark?slug=the-aggregate-trip-outcome-prediction-5-questions · How It Works · Data refreshed daily, snapshot 2026-11-05.