LLM-as-a-Reviewer: leaderboard
Large language models (LLMs) are increasingly used in academic peer review, yet their reliability, alignment with human judgment, and robustness to adversarial attacks remain poorly understood. We present a systematic benchmark of LLM-as-a-Reviewer on 898 papers stratified from NeurIPS and ICLR, evaluating 12 LLMs along three axes: rating calibration, divergence from human reviewers, and resistanc
Metric: Low (NeurIPS 2022) (self-reported). Source: benchmarklist.com. Saturation forecast: Around July 2027. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Haiku 4.5 | 99.7 | #280 |
| 2 | GPT-5.4 Mini | 99.7 | #212 |
| 3 | GPT-5.4 | 97.3 | #79 |
| 4 | GPT-5 Mini | 79.3 | #186 |
| 5 | GPT-4.1 Mini | 60.7 | #355 |
| 6 | Qwen 3 235B A22B | 48.3 | #313 |
| 7 | Gemini 3.1 Flash Lite | 48.3 | #191 |
| 8 | Gemini 2.5 Flash | 46.7 | #245 |
| 9 | Gemini 3 Flash | 44.3 | #99 |
| 10 | Qwen 3.5 9B | 29.3 | #379 |
| 11 | GPT-4o Mini | 28 | #599 |
| 12 | Llama 3.3 70B | 6.7 | #579 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=llm-as-a-reviewer · How It Works · Data refreshed daily, snapshot 2026-10-20.