ForecastBench-Sim Pandemic - Unconditional: leaderboard
Evaluates simulated pandemic forecasting over 30 scenarios at horizons 40 and 60. This series reports unconditional Brier loss, where lower is better, for the July 9, 2026 result snapshot. The conditional task supplies a vaccine intervention. Published means use scored responses (55–60 per model), retaining dated model identifiers. Source: the authors’ public ForecastBench-Sim result JSON.
Metric: Brier Score (0-1; scored responses). Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4.1 | 0.05 | #258 |
| 2 | Qwen 2.5 72B Instruct | 0.05 | #469 |
| 3 | Claude Haiku 4.5 (20251001) | 0.05 | #264 |
| 4 | Llama 3.1 70B Instruct | 0.05 | #567 |
| 5 | DeepSeek V3 | 0.05 | #329 |
| 6 | GPT-5 | 0.05 | #105 |
| 7 | GPT-5.1 (2025-11-13) | 0.06 | #107 |
| 8 | O3 Mini (2025-01-31) | 0.06 | #249 |
| 9 | O4 Mini (2025-04-16) | 0.07 | #183 |
| 10 | Gemini 2.5 Pro | 0.09 | #159 |
| 11 | GPT-5 Mini (2025-08-07) | 0.09 | #175 |
| 12 | GPT-5 Nano | 0.1 | #439 |
| 13 | O3 (2025-04-16) | 0.1 | #125 |
| 14 | Claude Sonnet 4.5 | 0.12 | #148 |
| 15 | Grok 4 | 0.19 | #189 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=forecastbench-sim-pandemic-unconditional · How It Works · Data refreshed daily, snapshot 2026-11-05.