ForecastGym - Last-state Accuracy (Tavily evidence): leaderboard

ForecastGym replays frozen web-search evidence for 150 resolved Polymarket and Kalshi questions (opened 14 September 2026) so every model forecasts from identical information: the score is the share of forecasts on the right side of 50% at each question's last evidence state, given Tavily's recorded results plus the model's own earlier forecasts and notes. Four models; the market's own price scores 71.9% on the same states.

Metric: Share of last-state forecasts on the correct side of 0.5 (%, read x100 from the printed 0-1 share; higher is better; a forecast of exactly 0.5 counts as half; ForecastGym first batch: 150 binary Polymarket and Kalshi questions opened 14 September 2026 and collected for 14 days, 614 frozen evidence states; the model reads each question's states offline in chronological order (main condition: Tavily's recorded evidence plus its previous forecast, rationale and persistent notes), one JSON-schema request per state with the same prompt and settings for every model; reasoning or thinking settings are not stated). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked. Updated 19 Nov 2026.

Top models

#ModelScoreOverall rank
1DeepSeek V4.1 Flash66#69
2GPT-5.6 Terra64.2#72
3Gemini 3.8 Flash62.8#43
4Claude Sonnet 559.7#118

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=forecastgym-last-state-accuracy-tavily-evidence · How It Works · Data refreshed daily, snapshot 2026-11-20.