TOC-Bench - Conditional State: leaderboard

The conditional state questions of TOC-Bench (2026), video questions grounded in per-frame object tracks and kept only when not answerable from text alone, single frames or shuffled frames: whether an object is fully visible, partly hidden, fully hidden or absent at the moment of another event. Accuracy in percent; higher is better.

Metric: Conditional State accuracy (%) on TOC-Bench (2,323 human-verified questions over 1,951 videos from Charades, Perception Test, OVIS and MOSE, grounded in per-frame object tracks and kept only when not answerable from text alone, single frames or shuffled frames), mostly 32 uniformly sampled frames, deterministic scoring of multiple-choice, statement-pair, numeric and ordering formats; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 21 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Kimi K2.656.5#192
2Grok 4.351.8#342
3Gemini 3.1 Pro (Preview)50#99
4Qwen 2.5 VL 7B43.9#1862
5Seed 2.0 Lite43.5#300
6MiMo-V2-Omni42.1#465
7GPT-5.541.7#54
8Qwen 3 VL 8B (Thinking)40.6#910
9Qwen 2.5 VL 72B40.6#781
10VideoChat-R1.5-7B40.6#1254
11Gemini 3.1 Flash Lite (Preview)39.6#417
12GLM-5V Turbo36.7#360
13GPT-5.4 Mini36.3#406
14Video-R135.3#1379
15InternVL3-8B31.7#1782

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=toc-bench-conditional-state · How It Works · Data refreshed daily, snapshot 2026-11-21.