SynopticBench: leaderboard
SynopticBench (2026) tests whether vision-language models can write weather forecast discussions from numerical weather prediction charts over North America, with National Weather Service area forecast discussions as references. This board is an LLM-judge score from 0 to 1 (Gemini 2.5 Flash judge) on the 2024 and 2025 test years; higher is better.
Metric: LLM-judge score (0-1, Gemini-2.5-Flash judge) of a generated weather forecast discussion against the National Weather Service area forecast discussion, from forecast-chart images over North America, 2024-2025 test years; higher is better. Source: arxiv.org. Saturation forecast: Around June 2027. 5 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 0.4185 | #99 |
| 2 | Llama 3.2 11B Vision (SynopticBench Checkpoint Unspecified) | 0.2623 | |
| 3 | Qwen 2.5 VL 7B (SynopticBench checkpoint unspecified) | 0.2566 | |
| 4 | LLaVA-v1.5-13B | 0.1896 | #6926 |
| 5 | LLaVA-v1.5-7B | 0.1565 | #7630 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=synopticbench · How It Works · Data refreshed daily, snapshot 2026-11-21.