Artifact-Bench (AI-Generated Video) - Artifact Identification: leaderboard

Artifact-Bench (2026) tests whether multimodal LLMs perceive the artifacts of AI-generated videos, on 1,100 manually annotated questions over real videos and clips from seven video generators, split into three difficulty levels. This task asks for every observable artifact type among six fine-grained candidates; accuracy is the mean of the three difficulty levels, in percent, higher is better.

Metric: Accuracy (%) on the 350 multi-answer questions asking for every observable artifact type among six fine-grained candidates (AID), the unweighted mean of the three difficulty levels, 1,100 manually annotated questions on 1,350 real and AI-generated videos from seven generators, at 5 fps frame sampling, each model at its officially recommended decoding (Gemini 3.1 Pro at temperature 1.0 and high thinking level) or greedy, answers parsed by Gemini 3.1 Pro; higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 19 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3.1 Pro (Preview)9.8#99
2Gemini 3 Flash9.8#175
3Qwen 3 VL 32B (Thinking)9.6#573
4Qwen 3 VL 30B A3B (Thinking)7.8#719
5Molmo2-8B7.8#1621
6VideoVeritas-8B7.8
7Qwen 3 VL 32B Instruct7.3#606
8Qwen 3 VL 30B A3B Instruct6.9#895
9MiMo-VL-7B-RL6.2#1155
10InternVL3.5-30B-A3B6.2#1267
11Qwen 3 VL 8B (Thinking)5.9#910
12Skyra-7B-RL5.8
13Skyra-7B-SFT5.7
14Qwen 3 VL 8B Instruct5.5#1029
15BusterX++ 7B5.3

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=artifact-bench-ai-generated-video-artifact-identification · How It Works · Data refreshed daily, snapshot 2026-11-21.