MobileGym-Bench - Deep-Dive Tasks: leaderboard
MobileGym-Bench (Wu et al., 2026) runs screenshot-only phone GUI agents in a browser-hosted simulation of 28 everyday and system apps, resets the simulator before each of 256 parameterized test tasks and judges success from the app state the agent leaves behind. This board covers the 68 deep-dive tasks, which drill down through several screens to reach the target. The score is the success rate in percent, averaged over four runs with re-sampled task parameters for open models and from one run for most proprietary models; higher is better.
Metric: Success rate (%; 68 of the 256 test tasks, state-checked; open models: mean of four runs with re-sampled task parameters; proprietary models: one run (two for Gemini 3.1 Pro)). Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 43.4 | #91 |
| 2 | Seed 2.0 Pro | 36.8 | #209 |
| 3 | Qwen 3.6 Plus | 32.4 | #217 |
| 4 | Qwen 3 VL 4B Instruct | 1.5 | #1281 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mobilegym-bench-deep-dive-tasks · How It Works · Data refreshed daily, snapshot 2026-11-12.