MT-JailBench - Crescendo: leaderboard
MT-JailBench (2026) benchmarks multi-turn jailbreak attacks under fixed budgets: an attacker model steers a conversation toward each of 159 HarmBench behaviors and a GPT-4o judge decides success. This board is the attack success rate in percent of the Crescendo attack against each target model; lower is better, meaning a more robust target.
Metric: Attack success rate (%) of the Crescendo multi-turn jailbreak (fixed-template escalating prompts, retry on refusal) over the 159 HarmBench behaviors used in prior multi-turn jailbreak work, black-box text-only, Qwen-2.5-32B as the attacker model, success judged by the GPT-4o 5-point Score-Judge, at most 5 turns and 20 target interactions with up to 3 retries per turn and no restarts; a lower rate means a more robust target, so lower is better. Source: arxiv.org. 21 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Haiku 4.5 | 21.38 | #583 | |
| 2 | GPT-5 | 25.16 | #178 | |
| 3 | Qwen 3.5 Flash | Thinking | 28.3 | #441 |
| 4 | Claude Sonnet 4.5 | Thinking | 30.82 | #261 |
| 5 | Qwen 3.5 Plus | Thinking | 36.48 | #242 |
| 6 | Nemotron 3 Super 120B A12B | Reasoning | 42.14 | |
| 7 | gemma-4-E4B-it | 48.43 | #1539 | |
| 8 | Grok 4.1 Fast | Reasoning | 49.69 | #427 |
| 9 | Llama 3 8B Instruct | 57.86 | #4852 | |
| 10 | Gemma 4 26B A4B (IT) | 58.49 | #556 | |
| 11 | Gemini 3 Pro | 62.89 | #145 | |
| 12 | Qwen 3 4B IT | 62.89 | ||
| 13 | Llama 4 Maverick | 63.52 | #1110 | |
| 14 | Llama 4 Scout | 64.78 | #1818 | |
| 15 | DeepSeek V3.2 | Thinking | 64.78 | #419 |
Interactive version: theaggregate.ai/benchmark?slug=mt-jailbench-crescendo · How It Works · Data refreshed daily, snapshot 2026-11-21.