AuthMem-Bench (Unauthorized Action Rate): leaderboard
AuthMem-Bench (Tsinghua University, 2026) tests whether an LLM agent respects the source authority of a consolidated memory when that memory has been laundered of its authorization metadata. The unauthorized-action rate is the share of non-authorizing cases in which the model issues the prohibited tool call with washed memory and no authority metadata, in percent, lower is better.
Metric: Attack success rate (%; share of 350 non-authorizing cases in which the action model issues the prohibited tool call, matched target tool and full argument object; washed focal memory with no authority metadata). Source: arxiv.org. Saturation forecast: Not forecast. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V4 Pro | 38.6 | #181 |
| 2 | Qwen 3.7 Max | 46.6 | #142 |
| 3 | Gemini 3.5 Flash | 52 | #98 |
| 4 | GLM-5.2 | 52 | #137 |
| 5 | Gemini 3.1 Pro (Preview) | 52.6 | #91 |
| 6 | GPT-5.5 | 54.3 | #48 |
| 7 | GPT-5.4 Mini (2026-03-17) | 56.3 | #285 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=authmem-bench-unauthorized-action-rate · How It Works · Data refreshed daily, snapshot 2026-11-12.