DUMA-Bench (Dual-Control): leaderboard
DUMA-Bench (ITMO University, 2026) extends tau2-bench with 35 adversarial agent scenarios in eight vulnerability classes. In this dual-control regime a GPT-4o-mini user simulator and the agent both act on the shared environment over several turns. Attack success rate in percent over repeated runs of each task, from executable state checks and GPT-4o-judged communication checks; lower is better.
Metric: Attack success rate (%; share of trials in which the agent violates a task's executable security assertions, plus GPT-4o-judged communication assertions on 9 of the 35 tau2-bench-derived tasks in 8 vulnerability domains; repeated independent runs of each task; dual-control regime: a GPT-4o-mini user simulator and the agent both act on the shared environment). Source: arxiv.org. Saturation forecast: Around February 2027. 14 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 8.6 | #244 |
| 2 | Claude Haiku 4.5 | 12.1 | #568 |
| 3 | Claude Opus 4.5 | 16.4 | #152 |
| 4 | GLM-4.7 | 35 | #365 |
| 5 | GPT-4.1 Mini | 35.7 | #775 |
| 6 | Qwen 3.5 Plus | 35.7 | #227 |
| 7 | GPT-4.1 | 38.6 | #492 |
| 8 | GPT-4o | 39.3 | #741 |
| 9 | DeepSeek V3.2 | 39.3 | #403 |
| 10 | Qwen 3.5 Flash | 42.1 | #443 |
| 11 | GPT-4o Mini | 56 | #1627 |
| 12 | GPT-5 | 64.7 | #163 |
| 13 | GPT-5 Mini | 72.9 | #349 |
| 14 | GPT-5 Nano | 79.3 | #888 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=duma-bench-dual-control · How It Works · Data refreshed daily, snapshot 2026-11-12.