MORSE-500: leaderboard
Each model answers all 500 programmatically generated videos, in which the question itself appears inside the video (API models such as o3 and Gemini 2.5 get frames at 2 fps, at most 32), covering abstract, math, physical, planning, spatial and temporal reasoning; the score is overall accuracy. Unlike video QA sets with text questions, the model gets only the minimal prompt "Answer the question in this video".
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 19 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | O3 (2025-04-16) | 23.6 | #229 |
| 2 | O4 Mini (2025-04-16) | 22.2 | #376 |
| 3 | Gemini 2.5 Pro (Preview 05-06) | 21.8 | #271 |
| 4 | O1 (2024-12-17) | 19.8 | #314 |
| 5 | Gemini 2.5 Flash (Preview 04-17) | 19.2 | #455 |
| 6 | Qwen 2.5 VL 72B Instruct | 17.8 | #849 |
| 7 | GPT-4o (2024-11-20) | 17.4 | #864 |
| 8 | Qwen2.5-VL-32B-Instruct-AWQ | 16.8 | |
| 9 | Qwen2.5-VL-72B-Instruct-AWQ | 16.4 | #704 |
| 10 | Gemini 2.0 Flash (001) | 16 | #727 |
| 11 | Qwen 2.5 VL 32B Instruct | 15.6 | #1074 |
| 12 | Gemma 3 27B (IT) | 14.6 | #1339 |
| 13 | Gemini 2.0 Flash Lite (001) | 14.2 | #1100 |
| 14 | MiniCPM-o-2.6 | 11.6 | #2417 |
| 15 | Qwen2.5-Omni-7B | 11.4 | #1883 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=morse-500 · How It Works · Data refreshed daily, snapshot 2026-11-17.