GeoBenchLLM - NY-POI: leaderboard
GeoBenchLLM (arXiv 2608.07411v1, Table 3): geo-spatial sub-datasets scored for Qwen3 0.6B, 1.7B and 8B with and without thinking and GPT-OSS 20B and 120B at low thinking budgets. Accuracy on NY-POI next point-of-interest recommendation from New York check-in histories, percent from two-decimal fractions. Higher is better.
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Around 2030. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-OSS-20B (Low) | 40 | #523 (GPT-OSS-20B) |
| 2 | Qwen 3 8B (Thinking) | 39 | #678 (Qwen 3 8B) |
| 3 | Qwen 3 1.7B (Non-reasoning) | 39 | #1175 (Qwen 3 1.7B) |
| 4 | GPT-OSS-120B (Low) | 39 | #346 (GPT-OSS-120B) |
| 5 | Qwen 3 8B (Non-reasoning) | 33 | #678 (Qwen 3 8B) |
| 6 | Qwen 3 0.6B (Thinking) | 33 | #1448 (Qwen 3 0.6B) |
| 7 | Qwen 3 1.7B (Thinking) | 33 | #1175 (Qwen 3 1.7B) |
| 8 | Qwen 3 0.6B (Non-reasoning) | 32 | #1448 (Qwen 3 0.6B) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=geobenchllm-ny-poi · How It Works · Data refreshed daily, snapshot 2026-11-05.