AgentVidBench (Single-Turn): leaderboard
AgentVidBench (KRAFTON, 2026) poses 100 multi-hop multiple-choice questions over 71 openly licensed videos, each needing evidence gathered from several moments of the video, with 25 distractor options per question. In the single-turn setting the model answers in one pass from sampled frames without tools. Accuracy in percent; higher is better.
Metric: Accuracy (%; 100 human-authored multi-hop multiple-choice questions over 71 openly licensed videos, each with the answer and 25 distractors, scored by exact option match; single pass without tools, 1 fps frame sampling except GPT-5 and Claude Opus 4.7 at 50 frames because of their API input limits). Source: arxiv.org. Saturation forecast: Around March 2028. 20 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 51 | #262 |
| 2 | Gemini 2.5 Flash | 46 | #452 |
| 3 | Gemma 4 31B | 38 | #393 |
| 4 | Gemma 4 31B (Reasoning) | 37 | #393 (Gemma 4 31B) |
| 5 | Claude Opus 4.7 | 30 | #75 |
| 6 | GPT-5 | 29 | #163 |
| 7 | Qwen 3.5 27B (Thinking) | 25 | #358 (Qwen 3.5 27B) |
| 8 | Qwen 3.5 27B (Non-reasoning) | 22 | #358 (Qwen 3.5 27B) |
| 9 | Qwen 3.5 9B (Thinking) | 22 | #803 (Qwen 3.5 9B) |
| 10 | Gemma 4 26B A4B | 21 | #617 |
| 11 | Qwen 3.5 9B (Non-reasoning) | 19 | #803 (Qwen 3.5 9B) |
| 12 | Qwen 3 VL 8B Instruct | 18 | #982 |
| 13 | Qwen 3 VL 8B (Thinking) | 18 | #887 |
| 14 | Qwen 3 VL 4B (Thinking) | 18 | #1303 |
| 15 | Qwen 3 VL 4B Instruct | 16 | #1281 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=agentvidbench-single-turn · How It Works · Data refreshed daily, snapshot 2026-11-12.