VideoHallucer: leaderboard

Video-language models answer yes/no questions on 948 videos in 1,000 pairs of a basic question and a hallucinated one, 200 pairs in each of five settings (object-relation, temporal, semantic detail, extrinsic factual, extrinsic non-factual); a pair counts only when both answers are right (1,800 distinct questions, since the two extrinsic settings share their basic questions). Scores are the paper's June 2024 runs on the original question set; in October 2025 the maintainers removed duplicate temporal questions, so the current release differs. Unlike image hallucination probes such as POPE, it tests videos and hallucinations that contradict world knowledge.

Metric: Overall pair accuracy over five hallucination settings (%). Source: arxiv.org. Saturation forecast: Around February 2027. 16 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1GPT-4o (2024-05-13)53.3#735
2PLLaVA-34B45#2441
3PLLaVA-13B41.2#3282
4PLLaVA-7B38.1#5042
5LLaVA-NeXT-Video-DPO (34B)32.3#2485
6LLaVA-NeXT-Video-7B-DPO32#8526
7LLaMA-VID-13B29.2
8MiniGPT4-Video-Mistral22.3
9LLaMA-VID-7B21#5333
10Video-LaVIT-v118.9
11Video-LLaVA-7B17.8#3875
12Video-LLaMA-2-7B10
13VideoChat2-7B7.8
14Video-ChatGPT6.4#6947
15Video-LLaMA-2-13B3.3

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=videohallucer · How It Works · Data refreshed daily, snapshot 2026-11-17.