DUMA-Bench (Solo): leaderboard
DUMA-Bench (ITMO University, 2026) extends tau2-bench with 35 adversarial agent scenarios in eight vulnerability classes, including retrieval poisoning, cross-agent manipulation, phishing campaigns and tool shadowing. In this passive-user regime the agent acts on the environment without an active user. Attack success rate in percent over repeated runs of each task, from executable state checks and GPT-4o-judged communication checks; lower is better.
Metric: Attack success rate (%; share of trials in which the agent violates a task's executable security assertions, plus GPT-4o-judged communication assertions on 9 of the 35 tau2-bench-derived tasks in 8 vulnerability domains; repeated independent runs of each task; passive-user regime: the agent acts on the environment without an active user). Source: arxiv.org. Saturation forecast: Estimated already saturated. 14 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 5 | #244 |
| 2 | Claude Opus 4.5 | 7.1 | #152 |
| 3 | Claude Haiku 4.5 | 7.9 | #568 |
| 4 | GPT-5 | 25.7 | #163 |
| 5 | GPT-5 Mini | 27.9 | #349 |
| 6 | GLM-4.7 | 27.9 | #365 |
| 7 | GPT-4.1 Mini | 29.3 | #775 |
| 8 | GPT-4o | 31.4 | #741 |
| 9 | GPT-5 Nano | 31.4 | #888 |
| 10 | Qwen 3.5 Flash | 32.9 | #443 |
| 11 | GPT-4.1 | 33.6 | #492 |
| 12 | DeepSeek V3.2 | 34.3 | #403 |
| 13 | Qwen 3.5 Plus | 39.3 | #227 |
| 14 | GPT-4o Mini | 44 | #1627 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=duma-bench-solo · How It Works · Data refreshed daily, snapshot 2026-11-12.