GeoBenchLLM - GeoSQA: leaderboard
GeoBenchLLM (arXiv 2608.07411v1, Table 3): geo-spatial sub-datasets scored for Qwen3 0.6B, 1.7B and 8B with and without thinking and GPT-OSS 20B and 120B at low thinking budgets. Accuracy on GeoSQA scenario-based geography multiple-choice questions, percent from two-decimal fractions. Higher is better.
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 8B (Thinking) | 63 | #678 (Qwen 3 8B) |
| 2 | GPT-OSS-120B (Low) | 57 | #346 (GPT-OSS-120B) |
| 3 | Qwen 3 8B (Non-reasoning) | 55 | #678 (Qwen 3 8B) |
| 4 | GPT-OSS-20B (Low) | 53 | #523 (GPT-OSS-20B) |
| 5 | Qwen 3 1.7B (Thinking) | 50 | #1175 (Qwen 3 1.7B) |
| 6 | Qwen 3 1.7B (Non-reasoning) | 38 | #1175 (Qwen 3 1.7B) |
| 7 | Qwen 3 0.6B (Thinking) | 37 | #1448 (Qwen 3 0.6B) |
| 8 | Qwen 3 0.6B (Non-reasoning) | 21 | #1448 (Qwen 3 0.6B) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=geobenchllm-geosqa · How It Works · Data refreshed daily, snapshot 2026-11-05.