ViMU - Open-Ended Interpretation: leaderboard

ViMU (2026) benchmarks video metaphorical understanding: whether models grasp the implicit, ironic or socially coded meaning of short online videos. This board is a rubric score in percent from an LLM judge for open-ended explanations of the intended meaning, penalizing hallucinated and purely literal answers; higher is better.

Metric: Open-ended interpretation score (%): an LLM judge rubric over core intent, implicit signal and social target, minus hallucination and literal-only penalties, rescaled to percent, hint-free questions on ViMU, short online videos whose meaning lies in subtext (irony, mockery, criticism), uniformly sampled frames, zero-shot through official implementations or APIs; questions and references written by GPT-5.4 and reviewed by human experts; higher is better. Source: arxiv.org. Saturation forecast: Around March 2027. 16 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.273.15#185
2GPT-5.4 Mini66.19#406
3O4 Mini65.27#368
4Qwen 3 VL 32B Instruct64.09#606
5MiMo-V2-Omni64.07#465
6Qwen 3.5 27B62.8#375
7Gemini 3 Flash (Preview)62.54#152
8GLM-4.5V62.52#754
9Seed 2.0 Lite60.84#300
10Grok 4.1 Fast57.62#427
11Gemma 3 27B (IT)55.9#1324
12Ministral 3 14B52.19#1870
13Claude 3 Haiku50.41#2663
14GPT-4.1 Nano50.12#2056
15Ministral 8B48.25#4857

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=vimu-open-ended-interpretation · How It Works · Data refreshed daily, snapshot 2026-11-21.