OmniGUI - Semantic Understanding: leaderboard
Metric: Exact match (%): share of steps whose predicted action type and parameters (tap point inside the target element, exact typed text) both match the expert action, on the 530 Semantic Understanding steps (textual, visual or spoken semantics) of OmniGUI (709 expert-demonstrated smartphone episodes, 2,579 action steps in 29 Chinese and English apps; at every step the agent receives the screenshot, a synchronous audio track and a video clip with the ground-truth action history), step-level teacher forcing, temperature 0; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 8 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Gemini 3 Pro | 61.1 |
| 2 | Gemini 3 Flash | 58.5 |
| 3 | Gemini 2.5 Pro | 37.7 |
| 4 | Gemini 2.5 Flash | 34.3 |
Interactive version: theaggregate.ai/benchmark?slug=omnigui-semantic-understanding · How It Works · Data refreshed daily, snapshot 2026-10-07.