OmniPro (Online): leaderboard

OmniPro (2026) evaluates proactive streaming video understanding: from continuous audio-visual streams of 2,700 human-verified questions, a model must decide when to respond and what to say across nine sub-tasks. In online mode a streaming model sees the instruction once and decides itself when to respond; this board is the F1 in percent of timely, correct responses against the triggers, averaged over the nine sub-tasks, higher is better.

Metric: Online-mode F1 (%): a streaming model sees the instruction at the start and decides itself when to respond; responses within 3 s of a ground-truth trigger and correct in content (exact match, or a Gemini-3-Flash rating of at least 3 of 5 for narration and step instruction) are matches; F1 of response precision and trigger recall, mean of the nine sub-tasks of OmniPro (2,700 human-verified proactive streaming questions on LongVALE and COIN videos, 84% needing audio; 1 fps input); higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1MiniCPM-o-4.520.9#1252
2MMDuet211.3
3LiveStar3.6#868

Interactive version: theaggregate.ai/benchmark?slug=omnipro-online · How It Works · Data refreshed daily, snapshot 2026-11-21.