GeoBenchLLM - StepGame: leaderboard
GeoBenchLLM (arXiv 2608.07411v1, Table 3): geo-spatial sub-datasets scored for Qwen3 0.6B, 1.7B and 8B with and without thinking and GPT-OSS 20B and 120B at low thinking budgets. Accuracy on StepGame multi-hop spatial reasoning over stated relations, percent from two-decimal fractions. Higher is better.
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-OSS-20B (Low) | 72 | #523 (GPT-OSS-20B) |
| 2 | GPT-OSS-120B (Low) | 64 | #346 (GPT-OSS-120B) |
| 3 | Qwen 3 8B (Thinking) | 53 | #678 (Qwen 3 8B) |
| 4 | Qwen 3 1.7B (Thinking) | 36 | #1175 (Qwen 3 1.7B) |
| 5 | Qwen 3 8B (Non-reasoning) | 23 | #678 (Qwen 3 8B) |
| 6 | Qwen 3 0.6B (Thinking) | 22 | #1448 (Qwen 3 0.6B) |
| 7 | Qwen 3 0.6B (Non-reasoning) | 15 | #1448 (Qwen 3 0.6B) |
| 8 | Qwen 3 1.7B (Non-reasoning) | 15 | #1175 (Qwen 3 1.7B) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=geobenchllm-stepgame · How It Works · Data refreshed daily, snapshot 2026-11-05.