TrustShiftProbe - Repository Management: leaderboard
Metric: Attack success rate (%, lower is better; share of the repository management attack sessions whose switched payload succeeds; each model as a ReAct agent connected to one MCP server that behaves benignly during a conditioning phase and then switches to an adversarial payload; 90 cases per domain; temperature 0, 15 tool calls; no defense). Source: arxiv.org. Saturation forecast: Around May 2028. 6 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-5 | 49.4 |
| 2 | O4 Mini (2025-04-16) | 58.2 |
| 3 | Claude Opus 4.8 | 76.2 |
| 4 | Qwen 3.5 Flash | 77 |
| 5 | GPT-4.1 | 78.4 |
| 6 | Grok 4.3 | 79.8 |
Interactive version: theaggregate.ai/benchmark?slug=trustshiftprobe-repository-management · How It Works · Data refreshed daily, snapshot 2026-09-29.