GeoAgentBench (Plan-and-Solve) - Tool Exact Match: leaderboard

GeoAgentBench (2026) is a dynamic execution benchmark for tool-augmented GIS agents: spatial analysis tasks run in a sandbox of 117 atomic GIS tools and are scored on the tool trajectory, the parameters of the executed calls and the produced map. This board is the percentage of the gold tool sequence matched exactly from the first step, with the agent using a full plan executed without feedback; higher is better.

Metric: Tools-Exact-Match (%): share of the ground-truth tool sequence matched exactly from the first step, on 53 spatial analysis tasks over 117 atomic GIS tools in an executable sandbox (30 steps per task), agents under the Plan-and-Solve paradigm (a full plan first, then execution without feedback); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Flash60.97#466
2GPT-4o58.18#754
3DeepSeek V351.99#736
4GPT-4o Mini46.23#1680
5Claude Sonnet 4.644.82#179
6Llama 3.1 8B Instruct28.67#4655
7Qwen 2.5 7B Instruct21.64#2732

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=geoagentbench-plan-and-solve-tool-exact-match · How It Works · Data refreshed daily, snapshot 2026-11-22.