Review Error Injection (Zero-Shot): leaderboard
A perturbation benchmark from Benchmarking Agentic Review Systems (2026): surface, claim, reasoning and experimental errors are injected into clean arXiv papers and a model reviews each paper in one call. The score is detection recall, the percent of injected errors that a comment quotes and correctly explains, on the 24-paper frontier subset; higher is better.
Metric: Error detection recall (%; share of injected errors that the review's comments quote and correctly explain, fuzzy quote match then a Gemini-3 Flash Preview explanation judge) on the 24-paper frontier subset of a perturbation benchmark built from 74 clean arXiv papers in 8 subject classes, with surface (math-token) and claim, reasoning and experimental errors injected; backends via OpenRouter without reasoning mode; zero-shot review (the model reads the corrupted paper and writes review comments in one call); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.5 (Non-reasoning) | 59.8 | #49 (GPT-5.5) |
| 2 | Claude Opus 4.7 | 54.3 | #79 |
| 3 | Grok 4.1 Fast (Non-reasoning) | 31.4 | #430 (Grok 4.1 Fast) |
| 4 | Qwen 3.6 35B A3B (Non-reasoning) | 31.4 | #462 (Qwen 3.6 35B A3B) |
| 5 | DeepSeek V4 Flash (Non-reasoning) | 31 | #258 (DeepSeek V4 Flash) |
| 6 | Gemini 3.1 Flash Lite (Non-reasoning) | 14.7 | #364 (Gemini 3.1 Flash Lite) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=review-error-injection-zero-shot · How It Works · Data refreshed daily, snapshot 2026-11-13.