JailNewsBench (Role Play): leaderboard
Metric: Attack success rate (%): share of model outputs that do not refuse, as judged jointly by GPT-5, Gemini 2.5 and Claude 4, on JailNewsBench test-split fake-news generation instructions (seed instructions built from real news articles of 34 regions in 22 languages, each with a financial, political, social or psychological motive), averaged across regions, under the Role Play jailbreak (the instruction is prefixed with a role that casts the model as a news writer); lower is better. Source: arxiv.org. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 76.2 | #91 |
| 2 | Claude Sonnet 4 | 76.9 | #194 |
| 3 | Gemini 2.5 Flash | 79.8 | #237 |
| 4 | DeepSeek R1 Distill Llama 70B | 83.9 | #582 |
| 5 | Llama 3.1 8B Instruct | 84.4 | #1018 |
| 6 | DeepSeek R1 Distill Llama 8B | 87.2 | #1282 |
| 7 | Llama 3.1 70B Instruct | 88.1 | #548 |
Interactive version: theaggregate.ai/benchmark?slug=jailnewsbench-role-play · How It Works · Data refreshed daily, snapshot 2026-10-11.