VideoReasonBench: leaderboard
The model watches an engine-generated video of 5 to 14 operations on a partly hidden state (numbers, cups, files, cards, chips) and answers questions that need the whole sequence: recall, infer, compare or predict states and operations; 1,440 questions per model, judged by Qwen2.5-72B. Unlike general video QA, answers depend on tracking state across the video rather than on one frame or world knowledge.
Metric: Overall accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 18 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 2.5 Pro (Preview 05-06) | 56 | #271 | |
| 2 | Gemini 2.5 Flash (Preview 04-17) | Thinking | 27.4 | #455 |
| 3 | Doubao-1.5-Thinking-Vision-Pro-250428 | 11.5 | #599 | |
| 4 | O4 Mini (2025-04-16) | 10.7 | #376 | |
| 5 | Gemini 2.0 Flash | 10.4 | #723 | |
| 6 | Qwen 2.5 VL 72B Instruct | 7.2 | #849 | |
| 7 | GPT-4o (2024-11-20) | 6.9 | #864 | |
| 8 | InternVL3-78B | 5.8 | #755 | |
| 9 | Kimi-VL-A3B-Instruct | 1.4 | #2180 | |
| 10 | Qwen 2.5 VL 7B Instruct | 1.3 | #1893 | |
| 11 | MiniCPM-V-2.6 | 0.8 | #3523 | |
| 12 | MiniCPM-o-2.6 | 0.8 | #2417 | |
| 13 | InternVL3-8B | 0.6 | #1756 | |
| 14 | llava-onevision-qwen2-7B-ov | 0.3 | #3252 | |
| 15 | LLaVA-Video-72B-Qwen2 | 0.1 | #1344 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=videoreasonbench · How It Works · Data refreshed daily, snapshot 2026-11-17.