25 games · October 2026

Guess the chess rating

How well can a model judge a player's established skill from one game? Every model gets the same anonymized moves and predicts both players' Lichess rapid ratings.

Model standings

Mean absolute error in rating points. Lower is better. All ranked models completed all 25 games.

RankModelMAEWithin 200Mean bias

The baselines are unranked. The held-out mean uses the other 24 games' ratings; it does not see either player in the held-out game. This small, deliberately broad sample cannot establish a reliable ordering between close scores.

Inspect every prediction

What the models were told

The target is explicit: each player's established pre-game Lichess rapid Glicko-2 rating, not FIDE, Chess.com, or a performance rating for this game.

Read the exact prompt

Sample and scoring

Point estimates and interval endpoints must be integers from 0 to 4,000, with estimates rounded to 50. MAE is the sum of the 50 absolute prediction errors divided by 50. Intervals and explanations are secondary outputs. The two estimates in each game are related; the reported descriptive uncertainty resamples whole games.

Prompt, sample, scoring and request settings were frozen before each batch. Opus 5.5's original pilot is reused. No prompts were adjusted after seeing the first results. Every model uses a separate call per game with no cross-game context. The request asks for medium reasoning with an 8,192-token output cap, including reasoning; medium has provider-specific meanings. Model explanations below are preserved as generated and are not independently verified chess annotations.

The 50 players each had at least 1,000 rapid games and rating deviation at most 60 at collection. Account-history checks were made near collection, not reconstructed for each historical game. A single game remains noisy evidence even when its players have established ratings. The original twenty games contain four games in each of five mean-rating bands; five further games deliberately add players rated below 1000; this is a convenience sample, not a representative sample of Lichess users.

Download games, prompt and results (JSON)Download predictions (CSV)

Source: Lichess API. About Lichess ratings.