Artifact-Bench (AI-Generated Video): leaderboard

Artifact-Bench (2026) tests whether multimodal LLMs perceive the artifacts of AI-generated videos, on 1,100 manually annotated questions over real videos and clips from seven video generators, split into three difficulty levels. The overall score is accuracy over all questions of its three tasks (real versus generated, pairwise realism, artifact identification); higher is better.

Metric: Overall accuracy (%) over all 1,100 questions of the three tasks (500 real vs AI-generated classification, 250 pairwise realism comparison, 350 artifact identification), 1,100 manually annotated questions on 1,350 real and AI-generated videos from seven generators, at 5 fps frame sampling, each model at its officially recommended decoding (Gemini 3.1 Pro at temperature 1.0 and high thinking level) or greedy, answers parsed by Gemini 3.1 Pro; higher is better. Source: arxiv.org. Saturation forecast: Around August 2028. 19 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3.1 Pro (Preview)47.5#99
2VideoVeritas-8B46
3Gemini 3 Flash43.8#175
4Qwen 3 VL 32B Instruct39.5#606
5Qwen 3 VL 32B (Thinking)37.3#573
6Qwen 3 VL 30B A3B (Thinking)36.3#719
7BusterX++ 7B36.2
8Qwen 3 VL 8B Instruct36#1029
9MiMo-VL-7B-RL35.7#1155
10Qwen 3 VL 30B A3B Instruct35.5#895
11MiMo-VL-7B-SFT35.4#1089
12InternVL3.5-38B34.7#1021
13Molmo2-8B34.5#1621
14InternVL3.5-8B34.5#1604
15InternVL3.5-30B-A3B33.8#1267

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=artifact-bench-ai-generated-video · How It Works · Data refreshed daily, snapshot 2026-11-21.