CLI-Advantage Suite (GUI Agents): leaderboard
The 45 Android task templates of the CLI-Advantage Suite (Gu et al., 2026), three seeds each, attempted by GUI agents that act on screenshots with taps, swipes and typing. Success rate in percent from rule-based verifiers of the final device state or answer, weighted by task; higher is better.
Metric: Success rate (%; 45 task templates x 3 seeds = 135 instances, rule-based verifier on the final device state or reported answer; mean of three seeds, weighted by task); screenshot-based GUI agents. Source: arxiv.org. Saturation forecast: Around June 2027. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 VL 32B Instruct | 22.2 | #582 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=cli-advantage-suite-gui-agents · How It Works · Data refreshed daily, snapshot 2026-11-12.