Forecasting resolved questions
Probabilities for 76 prediction-market questions that no model in the panel could look up from memory.
Run Sep 1, 2026, 76 items.
Method
Each question is a yes-or-no market on Manifold that resolved between May and August 2026. Before forecasting, every model is asked whether it already knows the outcome, and any question that any panel model answers correctly is dropped for everyone. That leaves 76 questions the models have to forecast rather than remember.
The score is the Brier score: the mean squared gap between the forecast probability and what happened, lower is better. Always answering 50% scores 0.250, and always answering the share of questions that resolved yes scores 0.249. No model beat that base rate: they were directionally right a little more often than chance and far too confident.
Results
| # | Model | Brier score (lower is better) | Items |
|---|---|---|---|
| Always the base rate | 0.249 | ||
| Always 50% | 0.250 | ||
| 1 | Claude Opus 5 | 0.273 | 76 |
| 2 | GPT-5.4 | 0.293 | 76 |
| 3 | Claude Opus 4.8 | 0.301 | 76 |
| 4 | GPT-5.5 | 0.305 | 76 |
| 5 | Claude Fable 5 | 0.311 | 75 |
| 6 | GPT-5.2 | 0.328 | 76 |
| 7 | DeepSeek V4 Pro | 0.330 | 75 |
| 8 | Kimi K2.6 | 0.335 | 75 |
| 9 | Qwen 3.6 Plus | 0.355 | 75 |
- Claude Fable 5.1 is not ranked. It was released after these questions resolved, and its edge shrinks the more recent the question, the sign of outcomes seen in training rather than forecasting skill.
In the main leaderboard
These results are collected for the main leaderboard alongside external benchmarks. New scores enter the rating through the regular audited data updates. Comparisons with that rating therefore share some of its inputs.
Interactive version: theaggregate.ai/our-benchmarks/forecasting · How It Works · Data refreshed daily, snapshot 2026-10-11.