Land or Water? (Open Models, 2° Reference): leaderboard

Favio Vazquez's text-only geographic knowledge evaluation: 16,200 text coordinates on a 2-degree grid, using the reference prompt with bare degree-and-hemisphere coordinates. Area-weighted accuracy weights correct answers by cos(latitude), on a 0–100% scale. The model receives only text latitude/longitude coordinates, with no images, maps, diagrams, video or tools. It identifies whether each location is land or water from geographic knowledge. The evaluator reads Land/Water next-token probabilities without generated reasoning; this is binary classification, not structured-output generation. Maps are plotted afterward by the evaluator. The answer key is Natural Earth 1:10m land v5.1.1; probability ties answer Water. Quantization and base/instruct settings are retained. Higher is better. These runs are separate from Celeste's Claude chart.

Metric: Area-weighted accuracy (%). Source: github.com. 1 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=land-or-water-open-models-2-reference · How It Works · Data refreshed daily, snapshot 2026-10-09.