Agent-SafetyBench: leaderboard
A tool-calling agent faces 2,000 cases in 349 simulated environments covering eight risk categories; the number is the share of interactions the ShieldAgent scorer labels safe. Unlike AgentHarm it mixes harmful requests with benign tasks where the risk comes from the environment.
Metric: Safety score (% safe, higher is better). Source: arxiv.org. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude 3 Opus | 59.8 | #1455 |
| 2 | Claude 3.5 Sonnet | 59.4 | #746 |
| 3 | Claude 3.5 Haiku | 55.1 | #1553 |
| 4 | GPT-4o | 44.2 | #741 |
| 5 | GPT-4 Turbo | 41.9 | #1001 |
| 6 | Gemini 1.5 Flash | 41.6 | #1668 |
| 7 | Gemini 1.5 Pro | 37.5 | #1049 |
| 8 | Qwen 2.5 72B Instruct | 37.3 | #1092 |
| 9 | GLM-4 9B Chat | 36.5 | #2759 |
| 10 | Llama 3.1 405B Instruct | 35.4 | #1115 |
| 11 | DeepSeek V2.5 | 34.2 | #1210 |
| 12 | Qwen 2.5 14B Instruct | 31.9 | #1732 |
| 13 | Llama 3.1 70B Instruct | 31.2 | #1546 |
| 14 | GPT-4o Mini | 31.2 | #1623 |
| 15 | Llama 3.1 8B Instruct | 19.9 | #4246 |
Interactive version: theaggregate.ai/benchmark?slug=agent-safetybench · How It Works · Data refreshed daily, snapshot 2026-11-13.