TOC-Bench - Hallucination Diagnostic Accuracy: leaderboard

Hallucination diagnostic accuracy of TOC-Bench (2026), video questions grounded in per-frame object tracks and kept only when not answerable from text alone, single frames or shuffled frames: accuracy on question variants about a nonexistent subject or a nonexistent event, where the model must reject a plausible but visually unsupported option. Percent; higher is better.

Metric: Hallucination diagnostic accuracy (%, HDA): accuracy on the nonexistent-subject and nonexistent-event question variants, where the model must reject a plausible but visually unsupported object or event, on TOC-Bench (2,323 human-verified questions over 1,951 videos from Charades, Perception Test, OVIS and MOSE, grounded in per-frame object tracks and kept only when not answerable from text alone, single frames or shuffled frames), mostly 32 uniformly sampled frames, deterministic scoring of multiple-choice, statement-pair, numeric and ordering formats; higher is better. Source: arxiv.org. Saturation forecast: Around September 2028. 21 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Grok 4.353.8#351
2Kimi K2.652.9#197
3Gemini 3.1 Pro (Preview)47.7#102
4Qwen 3 VL 8B (Thinking)43.8#924
5GPT-5.542.8#56
6Qwen 2.5 VL 72B42.6#782
7Gemini 3.1 Flash Lite (Preview)41.2#419
8Seed 2.0 Lite41.2#306
9Qwen 2.5 VL 7B35.3#1878
10MiMo-V2-Omni35.1#465
11GLM-5V Turbo34.7#365
12VideoChat-R1.5-7B33.1#1260
13Video-R132.5#1384
14GPT-5.4 Mini28.4#409
15InternVL3-8B27.8#1795

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=toc-bench-hallucination-diagnostic-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-22.