Beyond Rating - Weakness F1: leaderboard
Beyond Rating (2026) evaluates AI peer reviewers on the text of their reviews: generated reviews of ICLR and NeurIPS papers are split into atomic points and matched by Qwen3-235B against filtered high-confidence human reviews. This board is F1 in percent combining weakness Max-Recall with the share of the generated weakness points that match some human reviewer's point; higher is better.
Metric: F1 (from 0-1, times 100) combining point-wise precision (share of the generated review's weakness points matched by any human reviewer's weakness point) with weakness Max-Recall, on 1,000 test papers from ICLR 2024-2026 and NeurIPS 2022-2025 with three to five high-confidence human reviews each, the model writing a full review from the parsed paper; review points are split into atomic claims and matched to the human reviews' claims by Qwen3-235B; higher is better. Source: arxiv.org. Saturation forecast: Around 2031. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V3.2 | 31 | #419 |
| 2 | Claude Sonnet 4.5 | 29 | #261 |
| 3 | Qwen 3 235B A22B 2507 Instruct | 28 | #616 |
| 4 | AI Scientist + GPT-5 | 28 | |
| 5 | Gemini 3 Pro (Preview) | 26 | #119 |
| 6 | Qwen 3 30B A3B 2507 Instruct | 26 | #1100 |
| 7 | GPT-5.2 | 25 | #185 |
| 8 | DeepReviewer-14B | 25 | #844 |
| 9 | Qwen 3 8B | 24 | #1873 |
| 10 | SEA-E-7B | 23 | |
| 11 | AgentReview + GPT-5 | 23 | |
| 12 | CycleReviewer-70B | 22 | #1880 |
| 13 | OpenReviewer-8B | 21 | |
| 14 | Llama 3.1 70B Instruct | 18 | #1571 |
| 15 | Llama 3.1 8B Instruct | 17 | #4600 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=beyond-rating-weakness-f1 · How It Works · Data refreshed daily, snapshot 2026-11-21.