Land or Water? (Open Models, 10° Reference): leaderboard
Favio Vazquez's text-only geographic knowledge evaluation: 648 text coordinates on a 10-degree grid, using the reference prompt with bare degree-and-hemisphere coordinates. Area-weighted accuracy weights correct answers by cos(latitude), on a 0–100% scale. The model receives only text latitude/longitude coordinates, with no images, maps, diagrams, video or tools. It identifies whether each location is land or water from geographic knowledge. The evaluator reads Land/Water next-token probabilities without generated reasoning; this is binary classification, not structured-output generation. Maps are plotted afterward by the evaluator. The answer key is Natural Earth 1:10m land v5.1.1; probability ties answer Water. Quantization and base/instruct settings are retained. Higher is better. These runs are separate from Celeste's Claude chart.
Metric: Area-weighted accuracy (%). Source: github.com. Saturation forecast: Rough model projection: around 2026. 4 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-OSS-120B MXFP4 (Non-reasoning) | 75.68 |
| 2 | Gemma 4 26B A4B (IT) (Q8_0) (Non-reasoning) | 69.46 |
| 3 | Qwen 3.5 9B (Q8_0) (Non-reasoning) | 65.25 |
| 4 | Qwen 2.5 7B Instruct (Q8_0) | 48.47 |
Interactive version: theaggregate.ai/benchmark?slug=land-or-water-open-models-10-reference · How It Works · Data refreshed daily, snapshot 2026-10-09.