PaperAudit-Bench - Section-Level Detection@1: leaderboard
Finding errors injected into sections of ICML 2025 oral papers (PaperAudit-Bench, Tu et al., 2026): 1,209 corrupted sections with about 1.1 errors each, in eight categories such as evidence manipulation, method logic errors and claim distortion, with the model asked to report only the single error it is most confident about. F1 in percent, the harmonic mean of error coverage and finding precision, with a Qwen3-80B judge matching findings to the injected errors; higher is better.
Metric: F1 (%; harmonic mean of error coverage and finding precision on 1,209 corrupted sections of the ICML branch, the model prompted to report only its single most confident error; Qwen3-80B matching judge). Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 50.6 | #262 |
| 2 | Claude Sonnet 4.5 | 48.3 | #244 |
| 3 | Qwen 3 235B A22B | 45.9 | #622 |
| 4 | DeepSeek V3.1 | 30.6 | #500 |
| 5 | Qwen 3 14B | 18.5 | #1235 |
| 6 | Qwen 3 8B | 16.6 | #1838 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=paperaudit-bench-section-level-detection-1 · How It Works · Data refreshed daily, snapshot 2026-11-12.