ASPI: leaderboard
Clarification-seeking behavior is widely regarded as a desirable property of LLM agents, enabling them to resolve ambiguity before acting on underspecified tasks. However, the security implications of this interaction pattern remain unexplored. We investigate whether the transition from standard execution to a clarification-seeking state increases an agent's susceptibility to prompt injection atta
Metric: ASR exec_tool (self-reported). Source: benchmarklist.com. Saturation forecast: Around August 2027. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 0 | #79 |
| 2 | GPT-5.5 | 0 | #31 |
| 3 | Gemini 3.1 Pro (Preview) | 1.1 | #58 |
| 4 | O3 | 1.8 | #124 |
| 5 | Claude Opus 4.7 | 2.1 | #49 |
| 6 | Gemini 3 Flash | 2.2 | #99 |
| 7 | Kimi K2.5 | 11.1 | #144 |
| 8 | Qwen 3 32B | 18.7 | #443 |
| 9 | Qwen 3 235B A22B | 41.9 | #313 |
| 10 | DeepSeek V3.2 | 65.4 | #205 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=aspi · How It Works · Data refreshed daily, snapshot 2026-10-20.