JailNewsBench (Context Overload): leaderboard
Metric: Attack success rate (%): share of model outputs that do not refuse, as judged jointly by GPT-5, Gemini 2.5 and Claude 4, on JailNewsBench test-split fake-news generation instructions (seed instructions built from real news articles of 34 regions in 22 languages, each with a financial, political, social or psychological motive), averaged across regions, under the Context Overload jailbreak (512 tokens of unrelated news articles are prepended to the instruction); lower is better. Source: arxiv.org. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 76 | #91 |
| 2 | Claude Sonnet 4 | 78 | #194 |
| 3 | Gemini 2.5 Flash | 78.7 | #237 |
| 4 | DeepSeek R1 Distill Llama 70B | 85.1 | #582 |
| 5 | DeepSeek R1 Distill Llama 8B | 85.8 | #1282 |
| 6 | Llama 3.1 8B Instruct | 85.9 | #1018 |
| 7 | Llama 3.1 70B Instruct | 88.6 | #548 |
Interactive version: theaggregate.ai/benchmark?slug=jailnewsbench-context-overload · How It Works · Data refreshed daily, snapshot 2026-10-11.