CoCoReviewBench - Category Completeness: leaderboard
Metric: Category completeness (%): the average number of review categories the AI review covers per paper as a share of the union of categories all human reviewers of the paper cover; AI reviews of 1,300 ICLR and NeurIPS papers sampled from CoCoReviewBench (100 per year, original score distribution kept), each split into atomic opinions and compared category by category with the filtered human opinions by a GPT-5-Mini judge at temperature 0, general models prompted with the AI Scientist review prompt at temperature 0.6; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 18 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-5 Mini | 89.97 |
| 2 | GPT-5.2 | 84.49 |
| 3 | Nemotron 3 Nano 30B A3B (Reasoning) | 82.93 |
| 4 | QwQ-32B | 79.83 |
| 5 | Qwen 3 8B (Thinking) | 75.73 |
| 6 | Qwen 3 32B (Thinking) | 73.17 |
| 7 | Qwen 2.5 7B | 72.4 |
| 8 | Qwen 3 8B (Non-reasoning) | 72.28 |
| 9 | Gemini 3 Flash | 68.67 |
| 10 | Gemini 3 Pro | 67.69 |
| 11 | Llama 3.3 70B | 65.49 |
| 12 | Llama 3.1 8B | 62.07 |
Interactive version: theaggregate.ai/benchmark?slug=cocoreviewbench-category-completeness · How It Works · Data refreshed daily, snapshot 2026-10-07.