Artifact-Bench (AI-Generated Video) - Pairwise Realism Comparison: leaderboard

Artifact-Bench (2026) tests whether multimodal LLMs perceive the artifacts of AI-generated videos, on 1,100 manually annotated questions over real videos and clips from seven video generators, split into three difficulty levels. This task asks which of two AI-generated videos is more realistic; accuracy is the mean of the three difficulty levels, in percent, higher is better.

Metric: Accuracy (%) on the 250 questions asking which of two AI-generated videos is more realistic (PVRC), the unweighted mean of the three difficulty levels, 1,100 manually annotated questions on 1,350 real and AI-generated videos from seven generators, at 5 fps frame sampling, each model at its officially recommended decoding (Gemini 3.1 Pro at temperature 1.0 and high thinking level) or greedy, answers parsed by Gemini 3.1 Pro; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 19 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1VideoVeritas-8B53.1
2Gemini 3 Flash50.9#180
3Qwen 3 VL 32B Instruct50.8#607
4MiMo-VL-7B-SFT49.8#1097
5Gemini 3.1 Pro (Preview)48.6#102
6MiMo-VL-7B-RL46.9#1160
7InternVL3.5-8B46.5#1618
8Qwen 3 VL 8B Instruct45.8#1043
9Qwen 3 VL 30B A3B (Thinking)44.9#727
10Keye-VL-1.5-8B44.7#1729
11Qwen 3 VL 30B A3B Instruct44.6#903
12Qwen 3 VL 32B (Thinking)43.8#577
13InternVL3.5-38B42.9#1026
14BusterX++ 7B42.5
15Molmo2-8B40#1628

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=artifact-bench-ai-generated-video-pairwise-realism-comparison · How It Works · Data refreshed daily, snapshot 2026-11-22.