OmniGUI - Cross-modal Discrimination: leaderboard
OmniGUI (2026) is a step-level benchmark for smartphone GUI agents in omni-modal environments: at each of 2,579 steps in 709 expert episodes across 29 apps the agent receives a screenshot, synchronous audio and a video clip and predicts the next action, with the true action history given. This board is the step exact match in percent on the steps that require aligning video, audio and text cues; higher is better.
Metric: Exact match (%): share of steps whose predicted action type and parameters (tap point inside the target element, exact typed text) both match the expert action, on the 514 Cross-modal Discrimination steps (aligning video, audio and text cues) of OmniGUI (709 expert-demonstrated smartphone episodes, 2,579 action steps in 29 Chinese and English apps; at every step the agent receives the screenshot, a synchronous audio track and a video clip with the ground-truth action history), step-level teacher forcing, temperature 0; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Pro | 59.1 | #145 |
| 2 | Gemini 3 Flash | 56 | #175 |
| 3 | Gemini 2.5 Pro | 40.1 | #276 |
| 4 | Gemini 2.5 Flash | 35.7 | #463 |
| 5 | Qwen3-Omni | 26.2 | #1055 |
| 6 | VITA-1.5 | 11.5 | #3083 |
| 7 | MiniCPM-o-4.5 | 4.4 | #1252 |
| 8 | Baichuan-Omni-1.5 | 1.4 | #3285 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=omnigui-cross-modal-discrimination · How It Works · Data refreshed daily, snapshot 2026-11-21.