Review Error Injection (Zero-Shot): leaderboard
Metric: Error detection recall (%; share of injected errors that the review's comments quote and correctly explain, fuzzy quote match then a Gemini-3 Flash Preview explanation judge) on the 24-paper frontier subset of a perturbation benchmark built from 74 clean arXiv papers in 8 subject classes, with surface (math-token) and claim, reasoning and experimental errors injected; backends via OpenRouter without reasoning mode; zero-shot review (the model reads the corrupted paper and writes review comments in one call); higher is better. Source: arxiv.org. Saturation forecast: Around December 2026. 6 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-5.5 (Non-reasoning) | 59.8 |
| 2 | Claude Opus 4.7 | 54.3 |
| 3 | Grok 4.1 Fast (Non-reasoning) | 31.4 |
| 4 | Qwen 3.6 35B A3B (Non-reasoning) | 31.4 |
| 5 | DeepSeek V4 Flash (Non-reasoning) | 31 |
| 6 | Gemini 3.1 Flash Lite (Non-reasoning) | 14.7 |
Interactive version: theaggregate.ai/benchmark?slug=review-error-injection-zero-shot · How It Works · Data refreshed daily, snapshot 2026-09-29.