OmniPro - State Monitoring: leaderboard
Metric: Probe-mode accuracy (%): for each ground-truth trigger the model, given the video so far, is queried 2-5 s before (expecting a negative answer) and 0-3 s after (expecting the task answer, exact match on a structured output), and the trigger counts only if both are right, on the real-time state monitoring (report each transition of a monitored state) sub-task of OmniPro (2,700 human-verified proactive streaming questions on LongVALE and COIN videos, 84% needing audio; 1 fps input); higher is better. Source: arxiv.org. Saturation forecast: Around March 2028. 9 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Phi-4 Multimodal Instruct | 11.5 |
| 2 | Qwen2.5-Omni-7B | 8.6 |
Interactive version: theaggregate.ai/benchmark?slug=omnipro-state-monitoring · How It Works · Data refreshed daily, snapshot 2026-10-07.