Adversarial Humanities Benchmark: leaderboard
The model receives harmful requests from MLCommons AILuminate rewritten as humanities tasks (a literary analysis, a bureaucratic 'translation', a scholastic debate, a psychoanalysis of a memoir) that still ask for the harmful content; the number is the share answered unsafely by a three-judge majority. Unlike AILuminate, which asks plainly (3.84% pooled success in this paper), every prompt here hides the request inside a scholarly framing.
Metric: Attack success rate (%; share of humanities-style rewritten harmful requests answered with unsafe content, 3-judge majority vote, lower is better). Source: arxiv.org. 31 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 9.2 | #161 |
| 2 | Claude Opus 4.6 | 15.6 | #102 |
| 3 | GPT-5.4 Mini | 22.5 | #389 |
| 4 | Claude Haiku 4.5 | 25.5 | #568 |
| 5 | MiniMax-M2.7 | 29.2 | #518 |
| 6 | Qwen 3.5 27B | 29.3 | #359 |
| 7 | GPT-5.4 | 30.2 | #123 |
| 8 | Qwen 3.5 Flash (02-23) | 37 | #467 |
| 9 | Qwen 3.5 Plus (02-15) | 38.6 | #370 |
| 10 | Qwen 3.6 Plus | 41.1 | #222 |
| 11 | Grok 4.1 Fast | 54.2 | #430 |
| 12 | GPT-OSS-120B | 55.6 | #700 |
| 13 | GPT-OSS-20B | 59.5 | #1147 |
| 14 | Llama 4 Scout Instruct | 61.5 | #1543 |
| 15 | Seed 2.0 Lite | 62.5 | #289 |
Interactive version: theaggregate.ai/benchmark?slug=adversarial-humanities-benchmark · How It Works · Data refreshed daily, snapshot 2026-11-13.