Land or Water? (Open Models, 10° Degrees): leaderboard
Favio Vazquez's text-only geographic knowledge evaluation: 648 text coordinates on a 10-degree grid, using hemisphere letters with degree signs after a Coordinates label. Area-weighted accuracy weights correct answers by cos(latitude), on a 0–100% scale. The model receives only text latitude/longitude coordinates, with no images, maps, diagrams, video or tools. It identifies whether each location is land or water from geographic knowledge. The evaluator reads Land/Water next-token probabilities without generated reasoning; this is binary classification, not structured-output generation. Maps are plotted afterward by the evaluator. The answer key is Natural Earth 1:10m land v5.1.1; probability ties answer Water. Quantization and base/instruct settings are retained. Higher is better. These runs are separate from Celeste's Claude chart.
Metric: Area-weighted accuracy (%). Source: github.com. Saturation forecast: Rough model projection: around 2026. 6 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | OLMo 3.1 32B Instruct (Q8_0) | 73.8 |
| 2 | Qwen 3 4B (Q8_0) (Non-reasoning) | 71.23 |
| 3 | OLMo 3 7B Instruct (Q8_0) | 60.51 |
| 4 | Qwen 3 32B (Q8_0) (Non-reasoning) | 55.52 |
| 5 | Qwen 3 14B (Q8_0) (Non-reasoning) | 53.56 |
| 6 | OLMo 3 1025 7B (Q8_0) | 28.46 |
Interactive version: theaggregate.ai/benchmark?slug=land-or-water-open-models-10-degrees · How It Works · Data refreshed daily, snapshot 2026-10-09.