The Aggregate - Lichess Rating Prediction (25 Games): leaderboard
Predict each player’s pre-game Lichess rapid Glicko-2 rating from an anonymized standard 10+0 game. All 25 games involve 50 distinct established players, each with at least 1,000 rapid games and rating deviation at most 60 at collection. Every model sees the same frozen moves, result and time control, with no identities, ratings, engine analysis or tools. Medium reasoning is requested with an 8,192-token output cap. The score is mean absolute error across all 50 player estimates, in rating points; lower is better. Only complete 25-game panels are ranked. The original twenty games span five mean-rating bands, with five further games deliberately adding players rated below 1000; this convenience sample is not representative of the Lichess population. Reference predictors are not AI observations.
Metric: MAE (rating points). Source: theaggregate.ai. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) (Medium) | 269.66 | #54 (Gemini 3.1 Pro (Preview)) |
| 2 | Gemini 3 Flash (Preview) (Medium) | 271.82 | #78 (Gemini 3 Flash (Preview)) |
| 3 | Gemini 3.8 Flash (Medium) | 312.18 | #18 (Gemini 3.8 Flash) |
| 4 | Gemini 3.7 Flash (Medium) | 313.58 | #20 (Gemini 3.7 Flash) |
| 5 | GPT-6 Astra (Medium) | 334.02 | #6 (GPT-6 Astra) |
| 6 | GPT-6.1 Sol (Medium) | 335.38 | #9 (GPT-6.1 Sol) |
| 7 | Gemini 3.6 Flash (Medium) | 338.98 | #41 (Gemini 3.6 Flash) |
| 8 | Gemini 3.5 Flash (Medium) | 346.82 | #56 (Gemini 3.5 Flash) |
| 9 | GPT-5.6 Sol (Medium) | 371.94 | #16 (GPT-5.6 Sol) |
| 10 | GPT-6 Sol (Medium) | 399.74 | #14 (GPT-6 Sol) |
| 11 | Claude Opus 5.5 (Medium) | 438.26 | #3 (Claude Opus 5.5) |
| 12 | Claude Fable 5.1 (Medium) | 442.3 | #8 (Claude Fable 5.1) |
| 13 | GLM-5.3 Flash (Medium) | 464.38 | #61 (GLM-5.3 Flash) |
| 14 | GPT-5.6 Luna (Medium) | 464.9 | #66 (GPT-5.6 Luna) |
| 15 | Qwen3.8 Max Prime (Medium) | 480.66 |
Method and evidence
Run by The Aggregate · Oct 11, 2026
Read method and notes
25 rated 10+0 games, 50 distinct players. Every player had at least 1,000 rapid games, a non-provisional rating and rating deviation at most 60 at collection. Games were at most seven days old. Targets are the pre-game Lichess rapid Glicko-2 ratings, often informally called Elo.
The same frozen prompt tells every model exactly which rating to predict: Lichess rapid, not FIDE or Chess.com, separately for White and Black. It sees legal mainline moves, result and time control. Names, ratings, dates, game links, engine analysis and clock annotations are hidden. Models have no tools or cross-game context.
One call per game, medium reasoning requested and an 8,192-token output cap including reasoning. Provider meanings of medium can differ. Point estimates are rounded to 50; mean absolute error uses all 50 estimates. Predictions from the original ten games, the next ten and the five low-rated games are pooled without changes.
- Other 24 games' mean rating: 555.01
- Always predict 1500: 542.90
- 25 games, 50 player estimates per model. Lower mean absolute error is better. Only models completing all 25 games are ranked; every player estimate has equal weight.
- The first twenty games contain four games per original mean-rating band; five further games deliberately add players rated below 1000. This convenience sample is not representative of Lichess players. Both estimates from a game are related, and close scores do not establish a reliable ordering.
- Reference predictors are unranked. The held-out mean uses the other 24 games' ratings and sees neither player's rating from the held-out game.
- Recorded evaluation API cost: $8.1607. The displayed total also retains $0.0124 for requests without reported billing.
- Not ranked: DeepSeek V4.1 Flash completed 0 games in the original pilot and was not extended; see the evidence for the recorded request failure.
- Not ranked: Grok 4.7 completed 0 games in the original pilot and was not extended; see the evidence for the recorded request failure.
- Not ranked: GPT-5.6 Pro Sol completed 2 games in the original pilot and was not extended; see the evidence for the recorded request failure.
- Not ranked: GPT-5.6 Pro Luna completed 0 games in the original pilot and was not extended; see the evidence for the recorded request failure.
Interactive version: theaggregate.ai/benchmark?slug=the-aggregate-lichess-rating-prediction-25-games · How It Works · Data refreshed daily, snapshot 2026-10-12.