PhoneSafety - Harmful-Instruction Refusal: leaderboard
PhoneSafety scenario family (Tang et al., Tencent Hunyuan, 2026) in which the user's instruction itself asks the phone agent for something harmful, so the safe next action is to refuse. Safe-action rate in percent on 195 moments; higher is better.
Metric: Safe-action rate (%; 195 harmful-instruction refusal moments). Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 77.9 | #91 |
| 2 | Claude Opus 4.6 | 74.9 | #100 |
| 3 | Seed 2.0 Pro | 60 | #209 |
| 4 | Kimi K2.5 | 33.3 | #252 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=phonesafety-harmful-instruction-refusal · How It Works · Data refreshed daily, snapshot 2026-11-12.