Land or Water? (Open Models, 10° Hemisphere): leaderboard
Favio Vazquez's text-only geographic knowledge evaluation: 648 text coordinates on a 10-degree grid, using hemisphere letters after Latitude and Longitude labels. Area-weighted accuracy weights correct answers by cos(latitude), on a 0–100% scale. The model receives only text latitude/longitude coordinates, with no images, maps, diagrams, video or tools. It identifies whether each location is land or water from geographic knowledge. The evaluator reads Land/Water next-token probabilities without generated reasoning; this is binary classification, not structured-output generation. Maps are plotted afterward by the evaluator. The answer key is Natural Earth 1:10m land v5.1.1; probability ties answer Water. Quantization and base/instruct settings are retained. Higher is better. These runs are separate from Celeste's Claude chart.
Metric: Area-weighted accuracy (%). Source: github.com. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | OLMo 3 7B Instruct (Q8_0) | 60.79 |
| 2 | Qwen 3 4B (Q8_0) (Non-reasoning) | 56.05 |
| 3 | OLMo 3 1025 7B (Q8_0) | 28.46 |
Interactive version: theaggregate.ai/benchmark?slug=land-or-water-open-models-10-hemisphere · How It Works · Data refreshed daily, snapshot 2026-10-09.