MobileJudgeBench (Simple Baseline Judge): leaderboard
MobileJudgeBench (2026) measures how accurately an LLM judge decides from a recorded trajectory whether a mobile GUI agent completed its task, against human labels on 931 trajectories from six benchmarks. Accuracy in percent, higher is better, using the simple baseline judge proposed with the benchmark, which reads 48 sampled screenshots.
Metric: Judge accuracy (%; share of 931 human-annotated mobile-agent trajectories, from 6 benchmarks, 4 agent models and 68 apps, whose success or failure the judge classifies correctly; the authors' simple baseline judge: 48 uniformly sampled screenshots at a 600 px long edge, no UI metadata or agent reasoning, greedy decoding). Source: arxiv.org. Saturation forecast: Estimated already saturated. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 90.9 | #141 |
| 2 | GPT-5 Mini | 90.8 | #349 |
| 3 | Claude Sonnet 4.5 | 86 | #244 |
| 4 | Qwen 2.5 VL 72B Instruct | 85.7 | #845 |
| 5 | GLM-4.6V | 84.9 | #698 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mobilejudgebench-simple-baseline-judge · How It Works · Data refreshed daily, snapshot 2026-11-12.