ATBench - Real-World Harm Diagnosis: leaderboard

ATBench (2026) tests LLMs and guard models as safety judges of long-horizon agent tool-use trajectories built from a taxonomy of risk sources, failure modes and real-world harms. This board is the accuracy in percent of naming the real-world harm of each unsafe trajectory; higher is better.

Metric: Fine-grained diagnosis accuracy (%) of the real-world-harm label (ATBench taxonomy) over the 497 unsafe ATBench trajectories, with the AgentDoG diagnosis template; higher is better. Source: arxiv.org. Saturation forecast: Around 2035. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1AgentDoG-Qwen3-4B40.6
2GPT-5.430.2#135
3GPT-5.226.8#185
4Qwen 3 235B A22B 2507 Instruct26.6#616
5QwQ-32B22.9#907
6Qwen 3 4B 2507 Instruct21.2#2221
7Gemini 3.1 Pro (Preview)18.5#99
8Qwen 3 4B18.3#2348
9Llama 3.1 8B Instruct15.5#4600
10Qwen 2.5 7B Instruct15.5#2723
11Gemini 3 Flash15#175
12Qwen 3.5 4B8.2#1241
13Qwen 3.5 397B A17B6.8#264

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=atbench-real-world-harm-diagnosis · How It Works · Data refreshed daily, snapshot 2026-11-21.