LongPIBench - Code Review: leaderboard
Part of LongPIBench (Liu et al., 2026), which measures prompt injection hidden in long documents: the model reviews 100 long synthetic pull requests carrying an injected authority-spoofing instruction. Attack success rate in percent, the share of changes approved as the attacker asks, with no defense; lower is better.
Metric: Attack success rate (%; share of 100 long synthetic pull requests to review; success when the model approves the change, under the authority-spoofing prompt injection embedded in the context and the default attack goal, no defense; default inference settings, up to 20,000 output tokens (32,768 for Qwen3-8B)). Source: arxiv.org. Saturation forecast: Around 2034. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4.1 | 86 | #495 |
| 2 | Llama 3.1 8B Instruct | 87 | #4246 |
| 3 | Ministral-3-8B-Instruct-2512 | 88 | #1858 |
| 4 | GPT-4o | 89 | #741 |
| 5 | deepseek-llm-7B-chat | 89 | #7056 |
| 6 | Llama 3.2 3B Instruct | 91 | #7147 |
| 7 | Qwen 3 8B | 94 | #1830 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=longpibench-code-review · How It Works · Data refreshed daily, snapshot 2026-11-13.