ViMU - Open-Ended Interpretation: leaderboard
Metric: Open-ended interpretation score (%): an LLM judge rubric over core intent, implicit signal and social target, minus hallucination and literal-only penalties, rescaled to percent, hint-free questions on ViMU, short online videos whose meaning lies in subtext (irony, mockery, criticism), uniformly sampled frames, zero-shot through official implementations or APIs; questions and references written by GPT-5.4 and reviewed by human experts; higher is better. Source: arxiv.org. Saturation forecast: Around July 2027. 16 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-5.2 | 73.15 |
| 2 | GPT-5.4 Mini | 66.19 |
| 3 | O4 Mini | 65.27 |
| 4 | Qwen 3 VL 32B Instruct | 64.09 |
| 5 | MiMo-V2-Omni | 64.07 |
| 6 | Qwen 3.5 27B | 62.8 |
| 7 | Gemini 3 Flash (Preview) | 62.54 |
| 8 | GLM-4.5V | 62.52 |
| 9 | Seed 2.0 Lite | 60.84 |
| 10 | Grok 4.1 Fast | 57.62 |
| 11 | Gemma 3 27B (IT) | 55.9 |
| 12 | Ministral 3 14B | 52.19 |
| 13 | Claude 3 Haiku | 50.41 |
| 14 | GPT-4.1 Nano | 50.12 |
| 15 | Ministral 8B | 48.25 |
Interactive version: theaggregate.ai/benchmark?slug=vimu-open-ended-interpretation · How It Works · Data refreshed daily, snapshot 2026-10-07.