CLI-Advantage Suite (CLI Agents): leaderboard
Metric: Success rate (%; 45 task templates x 3 seeds = 135 instances, rule-based verifier on the final device state or reported answer; mean of three seeds, weighted by task); CLI coding agents over adb, bash only. Source: arxiv.org. Saturation forecast: Around December 2026. 3 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.7 (Claude Code) | 68.9 |
Interactive version: theaggregate.ai/benchmark?slug=cli-advantage-suite-cli-agents · How It Works · Data refreshed daily, snapshot 2026-09-26.