ATBench - Failure Mode Diagnosis: leaderboard

ATBench (2026) tests LLMs and guard models as safety judges of long-horizon agent tool-use trajectories built from a taxonomy of risk sources, failure modes and real-world harms. This board is the accuracy in percent of naming the failure mode of each unsafe trajectory; higher is better.

Metric: Fine-grained diagnosis accuracy (%) of the failure-mode label (ATBench taxonomy) over the 497 unsafe ATBench trajectories, with the AgentDoG diagnosis template; higher is better. Source: arxiv.org. Saturation forecast: Around 2034. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1AgentDoG-Qwen3-4B16.5
2GPT-5.413.5#138
3Gemini 3.1 Pro (Preview)12.6#102
4GPT-5.212#190
5Qwen 3 235B A22B 2507 Instruct11.6#625
6Qwen 3 4B 2507 Instruct9.6#2240
7QwQ-32B9.4#913
8Gemini 3 Flash8.3#180
9Qwen 3 4B8.2#2379
10Qwen 2.5 7B Instruct6#2732
11Llama 3.1 8B Instruct5.8#4655
12Qwen 3.5 397B A17B3.6#270
13Qwen 3.5 4B3#1270

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=atbench-failure-mode-diagnosis · How It Works · Data refreshed daily, snapshot 2026-11-22.