VideoReasonBench: leaderboard

The model watches an engine-generated video of 5 to 14 operations on a partly hidden state (numbers, cups, files, cards, chips) and answers questions that need the whole sequence: recall, infer, compare or predict states and operations; 1,440 questions per model, judged by Qwen2.5-72B. Unlike general video QA, answers depend on tracking state across the video rather than on one frame or world knowledge.

Metric: Overall accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 18 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 2.5 Pro (Preview 05-06)56#271
2Gemini 2.5 Flash (Preview 04-17)Thinking27.4#455
3Doubao-1.5-Thinking-Vision-Pro-25042811.5#599
4O4 Mini (2025-04-16)10.7#376
5Gemini 2.0 Flash10.4#723
6Qwen 2.5 VL 72B Instruct7.2#849
7GPT-4o (2024-11-20)6.9#864
8InternVL3-78B5.8#755
9Kimi-VL-A3B-Instruct1.4#2180
10Qwen 2.5 VL 7B Instruct1.3#1893
11MiniCPM-V-2.60.8#3523
12MiniCPM-o-2.60.8#2417
13InternVL3-8B0.6#1756
14llava-onevision-qwen2-7B-ov0.3#3252
15LLaVA-Video-72B-Qwen20.1#1344

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=videoreasonbench · How It Works · Data refreshed daily, snapshot 2026-11-17.