ViMU - Evidence Grounding: leaderboard

ViMU (2026) benchmarks video metaphorical understanding: whether models grasp the implicit, ironic or socially coded meaning of short online videos. This board is a multiple-choice score in percent for choosing the evidence sources that carry the meaning, with zero credit for any wrong selection and partial credit otherwise; higher is better.

Metric: Evidence grounding score (%): selecting the evidence sources (frames, on-screen text, editing, transcript, audio tone) that support the intended meaning, set-based multiple-choice scoring: 0 if any selected option is wrong, otherwise the share of gold options selected, averaged over items, on ViMU, short online videos whose meaning lies in subtext (irony, mockery, criticism), uniformly sampled frames, zero-shot through official implementations or APIs; questions and references written by GPT-5.4 and reviewed by human experts; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 16 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.267.83#185
2Seed 2.0 Lite66.16#300
3GPT-5.4 Mini64.45#406
4Grok 4.1 Fast63.84#427
5Qwen 3.5 27B60.28#375
6Qwen 3 VL 32B Instruct59.64#606
7O4 Mini59.63#368
8Ministral 3 14B55.73#1870
9Gemini 3 Flash (Preview)52.8#152
10Gemma 3 27B (IT)49.38#1324
11MiMo-V2-Omni48.94#465
12Ministral 8B48.6#4857
13Claude 3 Haiku34.55#2663
14Gemma 3 4B (IT)25.41#4139
15GLM-4.5V23.11#754

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=vimu-evidence-grounding · How It Works · Data refreshed daily, snapshot 2026-11-21.