Guess the chess rating
Estimate each player's established Lichess rapid rating from an anonymized game.
Run Oct 11, 2026, 20 items, $3.33 to run.
Method
Ten rated 10+0 games, twenty distinct players. Every player had at least 1,000 rapid games, a non-provisional rating and rating deviation at most 60 at collection. Games were at most seven days old. Targets are the pre-game Lichess rapid Glicko-2 ratings, often informally called Elo.
The same frozen prompt tells every model exactly which rating to predict: Lichess rapid, not FIDE or Chess.com, separately for White and Black. It sees legal mainline moves, result and time control. Names, ratings, dates, game links, engine analysis and clock annotations are hidden. Models have no tools or cross-game context.
One call per game, medium reasoning requested and an 8,192-token output cap including reasoning. Provider meanings of medium can differ. Point estimates are rounded to 50; mean absolute error uses all twenty estimates. The original Opus pilot is reused unchanged.
Results
| # | Model | MAE (rating points) (lower is better) |
|---|---|---|
| 1 | Gemini 3 Flash (Preview) (medium reasoning · 10 games) | 255.85 |
| 2 | Gemini 3.1 Pro (Preview) (medium reasoning · 10 games) | 263.95 |
| 3 | Gemini 3.8 Flash (medium reasoning · 10 games) | 280.85 |
| 4 | GPT-6.1 Sol (medium reasoning · 10 games) | 298.35 |
| 5 | Gemini 3.7 Flash (medium reasoning · 10 games) | 300.55 |
| 6 | GPT-6 Astra (medium reasoning · 10 games) | 302.35 |
| 7 | GPT-5.6 Sol (medium reasoning · 10 games) | 338.55 |
| 8 | Gemini 3.5 Flash (medium reasoning · 10 games) | 346.75 |
| 9 | GPT-6 Sol (medium reasoning · 10 games) | 352.05 |
| 10 | Gemini 3.6 Flash (medium reasoning · 10 games) | 357.05 |
| 11 | Claude Sonnet 5.5 (medium reasoning · 10 games) | 433.55 |
| 12 | GPT-5.6 Luna (medium reasoning · 10 games) | 433.55 |
| 13 | GLM-5.3 Flash (medium reasoning · 10 games) | 456.95 |
| 14 | Claude Opus 5.5 (medium reasoning · 10 games) | 463.35 |
| Other nine games' mean rating | 471.57 | |
| Always predict 1500 | 506.05 | |
| 15 | Claude Fable 5.1 (medium reasoning · 10 games) | 519.75 |
| 16 | Qwen3.8 Max Prime (medium reasoning · 10 games) | 523.05 |
- Pilot: 10 independent games, 20 player estimates. Lower mean absolute error is better. Only models completing all ten games are ranked.
- Two games per mean-rating band; this broad convenience sample is not representative of Lichess players. Both estimates from a game are related, and close scores do not establish a reliable ordering.
- Reference predictors are unranked. The held-out mean uses the other nine games' ratings and sees neither player's rating from the held-out game.
- Recorded evaluation API cost: $3.3134. The displayed total also retains $0.0124 for requests without reported billing.
- Not ranked: DeepSeek V4.1 Flash completed 0 of 10 games; see the evidence for the recorded request failure.
- Not ranked: Grok 4.7 completed 0 of 10 games; see the evidence for the recorded request failure.
- Not ranked: GPT-5.6 Pro Sol completed 2 of 10 games; see the evidence for the recorded request failure.
- Not ranked: GPT-5.6 Pro Luna completed 0 of 10 games; see the evidence for the recorded request failure.
Games, exact prompt and every prediction
In the main leaderboard
These results are collected for the main leaderboard alongside external benchmarks. New scores enter the rating through the regular audited data updates. Comparisons with that rating therefore share some of its inputs.
Interactive version: theaggregate.ai/our-benchmarks/lichess-rating · How It Works · Data refreshed daily, snapshot 2026-10-11.