MT-JailBench - XTeaming: leaderboard
MT-JailBench (2026) benchmarks multi-turn jailbreak attacks under fixed budgets: an attacker model steers a conversation toward each of 159 HarmBench behaviors and a GPT-4o judge decides success. This board is the attack success rate in percent of the XTeaming attack against each target model; lower is better, meaning a more robust target.
Metric: Attack success rate (%) of the XTeaming multi-turn jailbreak (planned attack strategies with TextGrad prompt refinement) over the 159 HarmBench behaviors used in prior multi-turn jailbreak work, black-box text-only, Qwen-2.5-32B as the attacker model, success judged by the GPT-4o 5-point Score-Judge, at most 5 turns and 20 target interactions with up to 3 retries per turn and no restarts; a lower rate means a more robust target, so lower is better. Source: arxiv.org. 21 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Haiku 4.5 | 0.63 | #583 | |
| 2 | Claude Sonnet 4.5 | Thinking | 3.14 | #261 |
| 3 | Qwen 3.5 Flash | Thinking | 23.27 | #441 |
| 4 | GPT-5 | 32.7 | #178 | |
| 5 | Nemotron 3 Super 120B A12B | Reasoning | 37.74 | |
| 6 | Qwen 3.5 Plus | Thinking | 39.62 | #242 |
| 7 | Gemini 3 Pro | 40.25 | #145 | |
| 8 | gemma-4-E4B-it | 46.45 | #1539 | |
| 9 | Llama 3 8B Instruct | 47.77 | #4852 | |
| 10 | Gemma 4 26B A4B (IT) | 47.8 | #556 | |
| 11 | Gemini 3 Flash | 53.46 | #175 | |
| 12 | Llama 4 Maverick | 55.97 | #1110 | |
| 13 | Nemotron 3 Nano | Reasoning | 57.86 | #2025 |
| 14 | Llama 4 Scout | 59.75 | #1818 | |
| 15 | Grok 4.1 Fast | Reasoning | 62.26 | #427 |
Interactive version: theaggregate.ai/benchmark?slug=mt-jailbench-xteaming · How It Works · Data refreshed daily, snapshot 2026-11-21.