ForecastBench-Sim Pandemic - Vaccine Conditional: leaderboard
Evaluates simulated pandemic forecasting over 30 scenarios at horizons 40 and 60. This series reports vaccine conditional Brier loss, where lower is better, for the July 9, 2026 result snapshot. The conditional task supplies a vaccine intervention. Published means use scored responses (55–60 per model), retaining dated model identifiers. Source: the authors’ public ForecastBench-Sim result JSON.
Metric: Brier Score (0-1; scored responses). Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4.1 | 0.02 | #258 |
| 2 | Claude Sonnet 4.5 | 0.04 | #148 |
| 3 | Llama 3.1 70B Instruct | 0.04 | #567 |
| 4 | GPT-5 Mini (2025-08-07) | 0.04 | #175 |
| 5 | GPT-5.1 (2025-11-13) | 0.05 | #107 |
| 6 | GPT-5 Nano | 0.07 | #439 |
| 7 | O3 Mini (2025-01-31) | 0.07 | #249 |
| 8 | DeepSeek V3 | 0.07 | #329 |
| 9 | Qwen 2.5 72B Instruct | 0.08 | #469 |
| 10 | GPT-5 | 0.09 | #105 |
| 11 | O3 (2025-04-16) | 0.1 | #125 |
| 12 | Claude Haiku 4.5 (20251001) | 0.1 | #264 |
| 13 | Gemini 2.5 Pro | 0.13 | #159 |
| 14 | O4 Mini (2025-04-16) | 0.15 | #183 |
| 15 | Grok 4 | 0.19 | #189 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=forecastbench-sim-pandemic-vaccine-conditional · How It Works · Data refreshed daily, snapshot 2026-11-05.