CoCoReviewBench - Grounding: leaderboard

CoCoReviewBench (2026) evaluates AI peer reviewers on ICLR and NeurIPS papers against category-specific human review opinions that were filtered for correctness using reviewer, author and meta-review discussions. This board measures how explicitly comments point to specific parts of the paper on a 1 to 5 judge scale, shown as the difference from the leave-one-out human reviewers, so positive values beat the humans; higher is better.

Metric: Category-level grounding score (how explicitly a comment points to a part of the paper and the issue in it) on the 1-5 judge scale, printed as the difference from the human reference reviews (3.75, leave-one-out: one human review scored against the others), so a positive value is better than the human reviewers and the range is -2.75 to +1.25; AI reviews of 1,300 ICLR and NeurIPS papers sampled from CoCoReviewBench (100 per year, original score distribution kept), each split into atomic opinions and compared category by category with the filtered human opinions by a GPT-5-Mini judge at temperature 0, general models prompted with the AI Scientist review prompt at temperature 0.6; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5.20.92#185
2GPT-5 Mini0.77#364
3Gemini 3 Flash0.69#175
4Gemini 3 Pro0.69#145
5QwQ-32B0.58#907
6Qwen 3 32BThinking0.43#972
7DeepReviewer-14B0.41#844
8DeepReviewer-7B0.18
9nemotron-3-nano-30B-a3BReasoning0.14#1946
10Qwen 2.5 7B-0.3#2929
11Qwen 3 8BThinking-0.32#1873
12Llama 3.1 8B-0.45#5196
13OpenReviewer-8B-0.52
14CycleReviewer-70B-0.55#1880
15Llama 3.3 70B-0.63#1514

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=cocoreviewbench-grounding · How It Works · Data refreshed daily, snapshot 2026-11-21.