AutoGUI-v2 - Element Captioning: leaderboard
AutoGUI-v2 (2026) tests functionality understanding of GUI regions and elements on web, desktop and mobile screenshots. This board asks which of several options states the outcome of an interaction with an element, with easy and hard distractors. Accuracy in percent; higher is better.
Metric: Element interaction-outcome captioning on 788 tasks, multiple-choice accuracy (%) with easy and hard (same-group) distractors; the authors' tuned prompts; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 70.3 | #276 |
| 2 | Claude Sonnet 4.5 | 66.9 | #261 |
| 3 | GPT-5 | 66.5 | #178 |
| 4 | O3 | 65.4 | #237 |
| 5 | GLM-4.5V | 64.8 | #754 |
| 6 | Qwen 3 VL 32B Instruct | 61.3 | #606 |
| 7 | Qwen 3 VL 8B Instruct | 57.1 | #1029 |
| 8 | Step-3 | 52.8 | #734 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=autogui-v2-element-captioning · How It Works · Data refreshed daily, snapshot 2026-11-21.