TrustShiftProbe - Browser Automation: leaderboard
TrustShiftProbe (2026) connects an LLM agent to an MCP tool server that first behaves honestly to build reliance and then switches to an adversarial payload, from malformed or withheld responses to schema-valid corrupted data and requests to leak secrets. This domain contains 90 browser automation test tasks. Score is attack success among sessions that consumed an adversarial response after the trust horizon, micro-averaged across exposed sessions without any defense, in percent; lower is better.
Metric: Attack success rate (%, lower is better; share of the browser automation attack sessions whose switched payload succeeds; each model as a ReAct agent connected to one MCP server that behaves benignly during a conditioning phase and then switches to an adversarial payload; 90 cases per domain; temperature 0, 15 tool calls; no defense). Source: arxiv.org. Saturation forecast: Not forecast. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Grok 4.3 | 62.3 | #333 |
| 2 | Qwen 3.5 Flash | 62.8 | #435 |
| 3 | GPT-5 | 65.4 | #167 |
| 4 | O4 Mini (2025-04-16) | 72 | #365 |
| 5 | GPT-4.1 | 76.5 | #495 |
| 6 | Claude Opus 4.8 | 85.1 | #56 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=trustshiftprobe-browser-automation · How It Works · Data refreshed daily, snapshot 2026-11-13.