RobotEQ-Video - Real Videos: leaderboard
The real subset of RobotEQ-Video (Che, Lian et al., 2026): 818 real egocentric clips filtered from existing video datasets (Ego4D, ExoEgo and CharadesEgo), with 5,625 candidate robot actions to judge proper or improper. The score is the mean of accuracy, macro-F1, probability of agreement and consensus-weighted agreement with the annotators, in percent; higher is better.
Metric: Real-subset Avg (%; action-properness judgment: given an egocentric robot-view video and a numbered list of candidate robot actions, the model labels each action proper or improper for socially appropriate proactive assistance; score is the mean of accuracy and macro-F1 against the majority-voted human label and of probability of agreement and consensus-weighted agreement with the four individual annotators; 818 real egocentric clips, 5,625 actions). Source: arxiv.org. Saturation forecast: Around May 2027. 48 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.6 Luna | 79.9 | #117 |
| 2 | GPT-5.5 | 78.7 | #48 |
| 3 | Gemini 3.5 Flash | 78.1 | #98 |
| 4 | Seed 2.0 Lite | 78.1 | #286 |
| 5 | Qwen 3.5 27B (Thinking) | 77.5 | #358 (Qwen 3.5 27B) |
| 6 | Gemini 3.1 Flash Lite | 77 | #363 |
| 7 | Kimi K2.6 | 75.5 | #173 |
| 8 | GLM-4.1V-9B (Thinking) | 75.3 | #1061 (GLM-4.1V-9B) |
| 9 | Gemma 4 12B (IT) | 75.3 | #688 |
| 10 | Qwen 3 VL 30B A3B Instruct | 75.2 | #870 |
| 11 | Qwen 3 VL 8B (Thinking) | 74.6 | #887 |
| 12 | Qwen 3 VL 8B Instruct | 74.5 | #982 |
| 13 | Gemma 4 31B (IT) | 74.5 | #343 |
| 14 | Qwen 3 VL 4B (Thinking) | 74.5 | #1303 |
| 15 | Qwen 3 VL 30B A3B (Thinking) | 74.2 | #710 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=roboteq-video-real-videos · How It Works · Data refreshed daily, snapshot 2026-11-12.