Beyond Rating - Rating MAE: leaderboard
Metric: Mean absolute error between the review's overall rating and the human reviewers' ratings on the ICLR scale (1, 3, 5, 6, 8, 10), on 1,000 test papers from ICLR 2024-2026 and NeurIPS 2022-2025 with three to five high-confidence human reviews each; lower is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 15 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | DeepSeek V3.2 | 0.98 |
| 2 | Qwen 3 235B A22B 2507 Instruct | 0.98 |
| 3 | Qwen 3 8B | 1.03 |
| 4 | Claude Sonnet 4.5 | 1.14 |
| 5 | GPT-5.2 | 1.14 |
| 6 | Qwen 3 30B A3B 2507 Instruct | 1.41 |
| 7 | Gemini 3 Pro (Preview) | 1.52 |
| 8 | Llama 3.1 70B Instruct | 2.32 |
| 9 | Llama 3.1 8B Instruct | 2.51 |
Interactive version: theaggregate.ai/benchmark?slug=beyond-rating-rating-mae · How It Works · Data refreshed daily, snapshot 2026-10-07.