PaperAudit-Bench - Section-Level Detection@all: leaderboard
Finding errors injected into sections of ICML 2025 oral papers (PaperAudit-Bench, Tu et al., 2026): 1,209 corrupted sections with about 1.1 errors each, in eight categories such as evidence manipulation, method logic errors and claim distortion, with the model free to report every error it finds. F1 in percent, the harmonic mean of error coverage and finding precision (so over-reporting costs precision), with a Qwen3-80B judge matching findings to the injected errors; higher is better.
Metric: F1 (%; harmonic mean of error coverage and finding precision on 1,209 corrupted sections of the ICML branch, the model free to report every error it finds; Qwen3-80B matching judge). Source: arxiv.org. Saturation forecast: Around 2032. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 14B | 42 | #1235 |
| 2 | Gemini 2.5 Pro | 40.1 | #262 |
| 3 | Qwen 3 8B | 39.5 | #1838 |
| 4 | Claude Sonnet 4.5 | 32.8 | #244 |
| 5 | Qwen 3 235B A22B | 32.6 | #622 |
| 6 | DeepSeek V3.1 | 29.1 | #500 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=paperaudit-bench-section-level-detection-all · How It Works · Data refreshed daily, snapshot 2026-11-12.