PhoneSafety - Over-Operation Protection: leaderboard
PhoneSafety scenario family (Tang et al., Tencent Hunyuan, 2026) in which the phone agent is about to do more than the user asked, and the safe action is to stop or back out of that path. Safe-action rate in percent on 170 moments; higher is better.
Metric: Safe-action rate (%; 170 over-operation protection moments). Source: arxiv.org. Saturation forecast: Around July 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Kimi K2.5 | 50.6 | #252 |
| 2 | Claude Opus 4.6 | 48.2 | #100 |
| 3 | Gemini 3.1 Pro (Preview) | 47.6 | #91 |
| 4 | Seed 2.0 Pro | 46.5 | #209 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=phonesafety-over-operation-protection · How It Works · Data refreshed daily, snapshot 2026-11-12.