OmniPro - State Monitoring: leaderboard
OmniPro (2026) evaluates proactive streaming video understanding: from continuous audio-visual streams of 2,700 human-verified questions, a model must decide when to respond and what to say across nine sub-tasks. In probe mode the model is queried just before and just after each trigger and scores only when both answers are right; this board is the probe accuracy in percent on reporting each transition of a monitored state, higher is better.
Metric: Probe-mode accuracy (%): for each ground-truth trigger the model, given the video so far, is queried 2-5 s before (expecting a negative answer) and 0-3 s after (expecting the task answer, exact match on a structured output), and the trigger counts only if both are right, on the real-time state monitoring (report each transition of a monitored state) sub-task of OmniPro (2,700 human-verified proactive streaming questions on LongVALE and COIN videos, 84% needing audio; 1 fps input); higher is better. Source: arxiv.org. Saturation forecast: Around September 2027. 8 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | MiniCPM-o-4.5 | 28.2 | #1251 |
| 2 | Qwen 3 Omni 30B A3B | 18.3 | #851 |
| 3 | Qwen 3 VL 8B | 18.2 | #927 |
| 4 | video-SALMONN-2+ 7B | 12.3 | #2329 |
| 5 | Phi-4 Multimodal Instruct | 11.5 | #4155 |
| 6 | Qwen2.5-Omni-7B | 8.6 | #1888 |
| 7 | InternVL3.5-8B | 7.2 | #1618 |
| 8 | VideoLLaMA2.1-7B-AV | 5.6 | #4199 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=omnipro-state-monitoring · How It Works · Data refreshed daily, snapshot 2026-11-22.