MT-JailBench - Crescendo: leaderboard

MT-JailBench (2026) benchmarks multi-turn jailbreak attacks under fixed budgets: an attacker model steers a conversation toward each of 159 HarmBench behaviors and a GPT-4o judge decides success. This board is the attack success rate in percent of the Crescendo attack against each target model; lower is better, meaning a more robust target.

Metric: Attack success rate (%) of the Crescendo multi-turn jailbreak (fixed-template escalating prompts, retry on refusal) over the 159 HarmBench behaviors used in prior multi-turn jailbreak work, black-box text-only, Qwen-2.5-32B as the attacker model, success judged by the GPT-4o 5-point Score-Judge, at most 5 turns and 20 target interactions with up to 3 retries per turn and no restarts; a lower rate means a more robust target, so lower is better. Source: arxiv.org. 21 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Haiku 4.521.38#583
2GPT-525.16#178
3Qwen 3.5 FlashThinking28.3#441
4Claude Sonnet 4.5Thinking30.82#261
5Qwen 3.5 PlusThinking36.48#242
6Nemotron 3 Super 120B A12BReasoning42.14
7gemma-4-E4B-it48.43#1539
8Grok 4.1 FastReasoning49.69#427
9Llama 3 8B Instruct57.86#4852
10Gemma 4 26B A4B (IT)58.49#556
11Gemini 3 Pro62.89#145
12Qwen 3 4B IT62.89
13Llama 4 Maverick63.52#1110
14Llama 4 Scout64.78#1818
15DeepSeek V3.2Thinking64.78#419

Interactive version: theaggregate.ai/benchmark?slug=mt-jailbench-crescendo · How It Works · Data refreshed daily, snapshot 2026-11-21.