MT-JailBench - XTeaming: leaderboard

MT-JailBench (2026) benchmarks multi-turn jailbreak attacks under fixed budgets: an attacker model steers a conversation toward each of 159 HarmBench behaviors and a GPT-4o judge decides success. This board is the attack success rate in percent of the XTeaming attack against each target model; lower is better, meaning a more robust target.

Metric: Attack success rate (%) of the XTeaming multi-turn jailbreak (planned attack strategies with TextGrad prompt refinement) over the 159 HarmBench behaviors used in prior multi-turn jailbreak work, black-box text-only, Qwen-2.5-32B as the attacker model, success judged by the GPT-4o 5-point Score-Judge, at most 5 turns and 20 target interactions with up to 3 retries per turn and no restarts; a lower rate means a more robust target, so lower is better. Source: arxiv.org. 21 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Haiku 4.50.63#583
2Claude Sonnet 4.5Thinking3.14#261
3Qwen 3.5 FlashThinking23.27#441
4GPT-532.7#178
5Nemotron 3 Super 120B A12BReasoning37.74
6Qwen 3.5 PlusThinking39.62#242
7Gemini 3 Pro40.25#145
8gemma-4-E4B-it46.45#1539
9Llama 3 8B Instruct47.77#4852
10Gemma 4 26B A4B (IT)47.8#556
11Gemini 3 Flash53.46#175
12Llama 4 Maverick55.97#1110
13Nemotron 3 NanoReasoning57.86#2025
14Llama 4 Scout59.75#1818
15Grok 4.1 FastReasoning62.26#427

Interactive version: theaggregate.ai/benchmark?slug=mt-jailbench-xteaming · How It Works · Data refreshed daily, snapshot 2026-11-21.