StepJack (Single-Step): leaderboard
Metric: Attack success rate (%, lower is better; share of examples whose environment-state checker finds the adversarial goal achieved; the goal injected as one instruction, k = 1; each computer-use agent in its default OSWorld agent implementation with 15 x k actions, on StepJack tasks in Reddit and OwnCloud environments with 12 adversarial goals; no urgency cue; one run). Source: arxiv.org. Saturation forecast: Around December 2027. 6 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Haiku 4.5 | 12.5 |
| 2 | Claude Sonnet 4.6 | 20.8 |
| 3 | Qwen 3.5 Plus | 29.2 |
| 4 | GPT-5.4 Mini (xHigh) | 41.7 |
| 5 | Kimi K2.5 | 52.1 |
Interactive version: theaggregate.ai/benchmark?slug=stepjack-single-step · How It Works · Data refreshed daily, snapshot 2026-09-29.