Omni2Web - Instruction Recovery: leaderboard
Omni2Web (Han et al., 2026) asks multimodal models to rewrite 918 bilingual screen recordings of weakly referential web-editing requests, where speech, cursor movement and edit history identify the targets into an ordered list of explicit targets and editing actions, without the source HTML. Instruction Recovery Score in percent is the mean of target F1 and action F1 after one-to-one matching with the reference instructions; higher is better.
Metric: Instruction Recovery Score (%; Track B: mean of target F1 and action F1 per instance, macro-averaged over 918 bilingual screen recordings of weakly referential webpage edit requests; the model rewrites the recording into an ordered list of explicit target and action instructions without the source HTML; one-to-one matching against reference instructions, semantic equivalence judged by Gemini 3.1 Pro; unusable outputs score zero). Source: arxiv.org. Saturation forecast: Around 2029. 17 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.5 Omni Plus | 49.14 | #237 |
| 2 | Claude Opus 5 | 48.91 | #20 |
| 3 | GPT-5.6 Sol | 48.34 | #29 |
| 4 | Gemini 3.5 Flash | 47.46 | #98 |
| 5 | Seed 2.0 Lite | 46.52 | #286 |
| 6 | Gemini 3.1 Pro (Preview) | 43.19 | #91 |
| 7 | Kimi K3 | 41.95 | #53 |
| 8 | Grok 4.6 | 40.83 | #44 |
| 9 | Seed 2.1 Pro | 39.08 | #133 |
| 10 | Qwen 3.8 Max | 38.37 | #55 |
| 11 | MiMo-V2.5 | 37.31 | #353 |
| 12 | Qwen 3.8 27B | 29.84 | #199 |
| 13 | MiniMax-M3 | 29.32 | #265 |
| 14 | Muse Spark 1.1 | 25.41 | #82 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=omni2web-instruction-recovery · How It Works · Data refreshed daily, snapshot 2026-11-12.