LiveGeo - Dynamic Shared Events: leaderboard

LiveGeo (arXiv 2611.17267v1, Table 3) tests geographic decisions with a shared ReAct agent scaffold and tool interface. Each task allows 200 interaction steps, temperature 0.1 and at most 10,000 generated tokens per turn, retaining the full interaction history. Results average three independent runs per task. The seven static categories contain 1,088 reviewed instances; the dynamic columns use 149 matched instances with the same replayed world timeline and events for every model. Task-specific judges enforce structured submissions and the documented checks. Dynamic success also requires observing the events, replanning correctly and passing at least 70% of applicable checks. The reported value is mean success as a percentage; confidence intervals are retained in the source. Appendix C.1 binds the exact API identities. Reasoning effort and downloadable weights are not inferred. World changes use shared-scope events. Higher is better.

Metric: Task-specific success rate (%). Source: arxiv.org. Saturation forecast: Around 2031. 9 models tracked. Updated 14 Nov 2026.

Top models

#ModelScoreOverall rank
1Gemini 3.7 Flash (Harness: LiveGeo ReAct)13.4
2Grok 4.5 (Harness: LiveGeo ReAct)6.7
3Qwen 3.8 Max (Harness: LiveGeo ReAct)4
4Gemini 3.1 Pro Preview (Harness: LiveGeo ReAct)3.4
5Kimi K32#56
6Claude Sonnet 5 (Harness: LiveGeo ReAct)2
7GPT-5.6 Sol1.3#30
8Claude Opus 5 (Harness: LiveGeo ReAct)1.3
9DeepSeek-V4-Flash-Vision-Exp (Harness: LiveGeo ReAct)1.3

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=livegeo-dynamic-shared-events · How It Works · Data refreshed daily, snapshot 2026-11-16.