MT-AgentRisk - Playwright: leaderboard
Metric: Attack success rate (%): share of the 140 Playwright-MCP web tasks (GitLab, OwnCloud, Reddit, shopping and shopping admin) the agent completes, on MT-AgentRisk multi-turn attack sequences (365 harmful tool-use tasks from OpenAgentSafety, SafeArena, P2SQL and harmful-converted MCPMark Notion tasks, each split by a taxonomy of addition and decomposition attacks into 2 to 7 innocuous-looking turns), each model as an OpenHands agent with real MCP tools at default settings and no defense; a GPT-4.1 judge labels each trajectory complete, rejected or failed; lower is better. Source: arxiv.org. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.2 (Medium) | 34.29 | #105 (GPT-5.2) |
| 2 | Claude Sonnet 4.5 | 65.71 | #138 |
| 3 | Qwen3 Coder | 67.86 | #285 |
| 4 | Seed-1.6 | 70.71 | #257 |
| 5 | Gemini 3 Flash | 72.86 | #93 |
| 6 | DeepSeek V3.2 | 80 | #198 |
Interactive version: theaggregate.ai/benchmark?slug=mt-agentrisk-playwright · How It Works · Data refreshed daily, snapshot 2026-10-11.