K9-Bench - Context Analysis: leaderboard
K9-Bench tests video understanding of domestic dogs: five-option questions about canine posture, action order, scene context, cause and effect, and interaction with people and other dogs, built from real-world web videos with an expert-grounded behaviour ontology. This board covers the questions about the scene and situation around the dog. Models answer in free form and an answer counts as correct when its embedding is closest to the gold option.
Metric: MCQ accuracy (%; the Context Analysis category of K9-Bench, 4,744 five-option questions on 907 real-world dog videos; the free-form CoT answer counts as correct when its Qwen3-Embedding-8B similarity to the gold option exceeds 0.5 and no wrong option is closer; 32 uniformly sampled frames for GPT-4o and the open models, the full video at 1 FPS for Gemini 2.5 Pro and Qwen3-VL-235B-A22B). Source: arxiv.org. Saturation forecast: Around March 2027. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 44 | #266 |
| 2 | Qwen 3 VL 235B A22B Instruct | 38.4 | #522 |
| 3 | Qwen 3 VL 32B Instruct | 35.4 | #585 |
| 4 | GPT-4o (2024-08-06) | 35 | #740 |
| 5 | Qwen 3 VL 32B (Thinking) | 33.7 | #559 |
| 6 | Qwen 3 VL 8B Instruct | 32.1 | #981 |
| 7 | Qwen 3 VL 4B Instruct | 30.8 | #1279 |
| 8 | Qwen 3 VL 8B (Thinking) | 29.7 | #887 |
| 9 | Qwen 3 VL 4B (Thinking) | 28 | #1296 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=k9-bench-context-analysis · How It Works · Data refreshed daily, snapshot 2026-11-13.