TOC-Bench: leaderboard

TOC-Bench (2026) tests temporal object consistency in video LLMs: 2,323 human-verified questions over 1,951 videos from Charades, Perception Test, OVIS and MOSE, grounded in per-frame object tracks, on whether objects appear, recur, keep their identity, move and change state over time. Overall accuracy in percent, mostly with 32 sampled frames; higher is better.

Metric: Overall accuracy (%) on TOC-Bench (2,323 human-verified questions over 1,951 videos from Charades, Perception Test, OVIS and MOSE, grounded in per-frame object tracks and kept only when not answerable from text alone, single frames or shuffled frames), mostly 32 uniformly sampled frames, deterministic scoring of multiple-choice, statement-pair, numeric and ordering formats; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 21 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.547.1#56
2Kimi K2.645#197
3Gemini 3.1 Pro (Preview)43.9#102
4Grok 4.343.8#351
5Seed 2.0 Lite42#306
6GLM-5V Turbo37.9#365
7Qwen 2.5 VL 72B34#782
8Gemini 3.1 Flash Lite (Preview)33.8#419
9GPT-5.4 Mini33.1#409
10Qwen 3 VL 8B (Thinking)32.1#924
11MiMo-V2-Omni31.6#465
12InternVL3-8B30.1#1795
13VideoChat-R1.5-7B29.4#1260
14LLaVA-Video-72B28.9#1211
15VideoLLaMA3-7B28.4#2064

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=toc-bench · How It Works · Data refreshed daily, snapshot 2026-11-22.