GeoAgentBench (Base Agent) - Tool Retrieval F1: leaderboard

GeoAgentBench (2026) is a dynamic execution benchmark for tool-augmented GIS agents: spatial analysis tasks run in a sandbox of 117 atomic GIS tools and are scored on the tool trajectory, the parameters of the executed calls and the produced map. This board is F1 in percent between the tools the agent invokes and the gold tool set, with the agent using zero-shot tool scheduling with sandbox execution feedback but no internal reasoning or error-recovery loop; higher is better.

Metric: Tools-Any-Order F1 (%) between the tools the agent invokes and the ground-truth tool set, on 53 spatial analysis tasks over 117 atomic GIS tools in an executable sandbox (30 steps per task), agents under the base agent paradigm (zero-shot tool scheduling with sandbox execution feedback but without an internal reasoning or error-recovery loop); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Flash82.48#466
2DeepSeek V379.78#736
3Claude Sonnet 4.674.7#179
4GPT-4o68.95#754
5GPT-4o Mini68.27#1680
6Llama 3.1 8B Instruct52.12#4655
7Qwen 2.5 7B Instruct22.01#2732

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=geoagentbench-base-agent-tool-retrieval-f1 · How It Works · Data refreshed daily, snapshot 2026-11-22.