ViMU - Social Value Signal Identification: leaderboard

ViMU (2026) benchmarks video metaphorical understanding: whether models grasp the implicit, ironic or socially coded meaning of short online videos. This board is a multiple-choice score in percent for identifying the social values the video signals, with zero credit for any wrong selection and partial credit otherwise; higher is better.

Metric: Social value signal identification score (%): selecting the social values or stances the video signals, set-based multiple-choice scoring: 0 if any selected option is wrong, otherwise the share of gold options selected, averaged over items, on ViMU, short online videos whose meaning lies in subtext (irony, mockery, criticism), uniformly sampled frames, zero-shot through official implementations or APIs; questions and references written by GPT-5.4 and reviewed by human experts; higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 16 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1O4 Mini29.51#372
2Grok 4.1 Fast28.73#431
3Gemini 3 Flash (Preview)28.26#156
4Qwen 3.5 27B22.4#380
5GPT-5.221.15#190
6MiMo-V2-Omni18.52#465
7Seed 2.0 Lite16.73#306
8Qwen 3 VL 32B Instruct15.17#607
9GPT-5.4 Mini11.77#409
10Ministral 8B10.45#4866
11GLM-4.5V9.26#756
12GPT-4.1 Nano9.02#2065
13Gemma 3 27B (IT)7.95#1341
14Gemma 3 4B (IT)7.17#4227
15Ministral 3 14B6.57#1885

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=vimu-social-value-signal-identification · How It Works · Data refreshed daily, snapshot 2026-11-22.