LongPIBench - Email Summary: leaderboard
Part of LongPIBench (Liu et al., 2026), which measures prompt injection hidden in long documents: the model summarizes 100 long synthetic email threads and drafts a reply while an authority-spoofing instruction is injected. Attack success rate in percent, the share of summaries that include the link of the attacker, with no defense; lower is better.
Metric: Attack success rate (%; share of 100 long synthetic email threads to summarize with a draft reply; success when the summary contains the attacker's link, under the authority-spoofing prompt injection embedded in the context and the default attack goal, no defense; default inference settings, up to 20,000 output tokens (32,768 for Qwen3-8B)). Source: arxiv.org. Saturation forecast: Around 2029. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4.1 | 68 | #495 |
| 2 | GPT-4o | 69 | #741 |
| 3 | Llama 3.1 8B Instruct | 70 | #4246 |
| 4 | Ministral-3-8B-Instruct-2512 | 77 | #1858 |
| 5 | deepseek-llm-7B-chat | 79 | #7056 |
| 6 | Llama 3.2 3B Instruct | 83 | #7147 |
| 7 | Qwen 3 8B | 97 | #1830 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=longpibench-email-summary · How It Works · Data refreshed daily, snapshot 2026-11-13.