SPADE-Bench: leaderboard
SPADE-Bench (2026) tests whether tool-using LLM agents report plans that diverge from the actions they execute when a scenario puts them under pressure, using 300 paired regular and pressure scenarios over 239 simulated tools. The score is the English deception rate: the percent of executed cases in which any of five samples shows a stated plan that shifts toward the external incentive while the tool action does not, as judged by a released Qwen3-32B stance judge. Lower is better.
Metric: Deception rate (%, Pass@5; lower is better): share of the executed SPADE-Bench cases (300 paired regular and pressure scenarios over 239 simulated tools, English prompts) in which any of 5 samples at temperature 0.7 shows a plan that shifts toward the external incentive under pressure while the executed tool action does not, judged by the released Qwen3-32B stance judge; cases the model fails to execute are left out of the denominator. Source: arxiv.org. Saturation forecast: Not forecast. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3.3 70B Instruct | 24.91 | #1357 |
| 2 | GPT-5.1 | 25 | #229 |
| 3 | Kimi K2 | 30 | #439 |
| 4 | Claude Sonnet 4.5 | 33.11 | #250 |
| 5 | GLM-4.6 | 36.95 | #486 |
| 6 | DeepSeek V3.1 | 39 | #497 |
| 7 | Qwen 3 32B | 43.62 | #960 |
| 8 | Gemini 2.5 Pro | 57.33 | #266 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=spade-bench · How It Works · Data refreshed daily, snapshot 2026-11-13.