Beyond Rating - Weakness F1: leaderboard

Beyond Rating (2026) evaluates AI peer reviewers on the text of their reviews: generated reviews of ICLR and NeurIPS papers are split into atomic points and matched by Qwen3-235B against filtered high-confidence human reviews. This board is F1 in percent combining weakness Max-Recall with the share of the generated weakness points that match some human reviewer's point; higher is better.

Metric: F1 (from 0-1, times 100) combining point-wise precision (share of the generated review's weakness points matched by any human reviewer's weakness point) with weakness Max-Recall, on 1,000 test papers from ICLR 2024-2026 and NeurIPS 2022-2025 with three to five high-confidence human reviews each, the model writing a full review from the parsed paper; review points are split into atomic claims and matched to the human reviews' claims by Qwen3-235B; higher is better. Source: arxiv.org. Saturation forecast: Around 2031. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1DeepSeek V3.231#419
2Claude Sonnet 4.529#261
3Qwen 3 235B A22B 2507 Instruct28#616
4AI Scientist + GPT-528
5Gemini 3 Pro (Preview)26#119
6Qwen 3 30B A3B 2507 Instruct26#1100
7GPT-5.225#185
8DeepReviewer-14B25#844
9Qwen 3 8B24#1873
10SEA-E-7B23
11AgentReview + GPT-523
12CycleReviewer-70B22#1880
13OpenReviewer-8B21
14Llama 3.1 70B Instruct18#1571
15Llama 3.1 8B Instruct17#4600

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=beyond-rating-weakness-f1 · How It Works · Data refreshed daily, snapshot 2026-11-21.