Beyond Rating - Rating MAE: leaderboard

Beyond Rating (2026) evaluates AI peer reviewers on the text of their reviews: generated reviews of ICLR and NeurIPS papers are split into atomic points and matched by Qwen3-235B against filtered high-confidence human reviews. This board is the mean absolute error of the review's overall rating against the human ratings on the ICLR scale; lower is better.

Metric: Mean absolute error between the review's overall rating and the human reviewers' ratings on the ICLR scale (1, 3, 5, 6, 8, 10), on 1,000 test papers from ICLR 2024-2026 and NeurIPS 2022-2025 with three to five high-confidence human reviews each; lower is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1DeepReviewer-14B0.85#844
2OpenReviewer-8B0.86
3DeepSeek V3.20.98#419
4Qwen 3 235B A22B 2507 Instruct0.98#616
5Qwen 3 8B1.03#1873
6SEA-E-7B1.1
7Claude Sonnet 4.51.14#261
8GPT-5.21.14#185
9AgentReview + GPT-51.28
10CycleReviewer-70B1.29#1880
11Qwen 3 30B A3B 2507 Instruct1.41#1100
12Gemini 3 Pro (Preview)1.52#119
13Llama 3.1 70B Instruct2.32#1571
14Llama 3.1 8B Instruct2.51#4600
15AI Scientist + GPT-52.68

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=beyond-rating-rating-mae · How It Works · Data refreshed daily, snapshot 2026-11-21.