LongPIBench - Code Review: leaderboard
Metric: Attack success rate (%; share of 100 long synthetic pull requests to review; success when the model approves the change, under the authority-spoofing prompt injection embedded in the context and the default attack goal, no defense; default inference settings, up to 20,000 output tokens (32,768 for Qwen3-8B)). Source: arxiv.org. Saturation forecast: Around 2036. 8 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-4.1 | 86 |
| 2 | Llama 3.1 8B Instruct | 87 |
| 3 | GPT-4o | 89 |
| 4 | deepseek-llm-7B-chat | 89 |
| 5 | Llama 3.2 3B Instruct | 91 |
| 6 | Qwen 3 8B | 94 |
Interactive version: theaggregate.ai/benchmark?slug=longpibench-code-review · How It Works · Data refreshed daily, snapshot 2026-09-29.