SPADE-Bench (Chinese): leaderboard
The deception rate of SPADE-Bench (2026) with its 300 paired pressure scenarios presented in Chinese: the percent of executed cases in which any of five samples shows a stated plan that shifts toward the external incentive while the executed tool action does not. Lower is better.
Metric: Deception rate (%, Pass@5; lower is better): share of the executed SPADE-Bench cases (300 paired regular and pressure scenarios over 239 simulated tools, Chinese prompts) in which any of 5 samples at temperature 0.7 shows a plan that shifts toward the external incentive under pressure while the executed tool action does not, judged by the released Qwen3-32B stance judge; cases the model fails to execute are left out of the denominator. Source: arxiv.org. Saturation forecast: Around January 2028. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3.3 70B Instruct | 18.32 | #1357 |
| 2 | GLM-4.6 | 35.12 | #486 |
| 3 | Qwen 3 32B | 42.47 | #960 |
| 4 | Gemini 2.5 Pro | 46.49 | #266 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=spade-bench-chinese · How It Works · Data refreshed daily, snapshot 2026-11-13.