Guess the chess rating

Model standings

Mean absolute error in rating points. Lower is better. All ranked models completed all ten games.

The baselines are unranked. The held-out mean uses the other nine games' ratings; it does not see either player in the held-out game. This small, deliberately broad sample cannot establish a reliable ordering between close scores.

Inspect every prediction

What the models were told

The target is explicit: each player's established pre-game Lichess rapid Glicko-2 rating, not FIDE, Chess.com, or a performance rating for this game.

Sample and scoring

Point estimates and interval endpoints must be integers from 0 to 4,000, with estimates rounded to 50. MAE is the sum of the twenty absolute prediction errors divided by twenty. Intervals and explanations are secondary outputs. The two estimates in each game are related; the reported descriptive uncertainty resamples whole games.

Prompt, sample, scoring and request settings were frozen before each batch. Opus 5.5's original pilot is reused. No prompts were adjusted after seeing the first results. Every model uses a separate call per game with no cross-game context. The request asks for medium reasoning with an 8,192-token output cap, including reasoning; medium has provider-specific meanings. Model explanations below are preserved as generated and are not independently verified chess annotations.

The 20 players each had at least 1,000 rapid games and rating deviation at most 60 at collection. Account-history checks were made near collection, not reconstructed for each historical game. A single game remains noisy evidence even when its players have established ratings. Two games were chosen in each of five mean-rating bands; this is a convenience sample, not a representative sample of Lichess users.

Download games, prompt and results (JSON)Download predictions (CSV)

Source: Lichess API. About Lichess ratings.