OmniPro - Step Instruction: leaderboard

OmniPro (2026) evaluates proactive streaming video understanding: from continuous audio-visual streams of 2,700 human-verified questions, a model must decide when to respond and what to say across nine sub-tasks. In probe mode the model is queried just before and just after each trigger and scores only when both answers are right; this board is the probe accuracy in percent on giving the next step of a procedural task at the right moment, higher is better.

Metric: Probe-mode accuracy (%): for each ground-truth trigger the model, given the video so far, is queried 2-5 s before (expecting a negative answer) and 0-3 s after (expecting the task answer, exact match on a structured output), and the trigger counts only if both are right, on the sequential step instruction (next-step guidance in a procedural task, scored as multiple choice) sub-task of OmniPro (2,700 human-verified proactive streaming questions on LongVALE and COIN videos, 84% needing audio; 1 fps input); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3 Omni 30B A3B31.6#847
2Qwen 3 VL 8B25.8#917
3MiniCPM-o-4.525.8#1252
4InternVL3.5-8B20#1604
5Qwen2.5-Omni-7B18.2#1884
6Phi-4 Multimodal Instruct16.9#4154
7video-SALMONN-2+ 7B15.6#2322
8VideoLLaMA2.1-7B-AV14#4204

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=omnipro-step-instruction · How It Works · Data refreshed daily, snapshot 2026-11-21.