Artifact-Bench (AI-Generated Video) - Artifact Identification: leaderboard
Artifact-Bench (2026) tests whether multimodal LLMs perceive the artifacts of AI-generated videos, on 1,100 manually annotated questions over real videos and clips from seven video generators, split into three difficulty levels. This task asks for every observable artifact type among six fine-grained candidates; accuracy is the mean of the three difficulty levels, in percent, higher is better.
Metric: Accuracy (%) on the 350 multi-answer questions asking for every observable artifact type among six fine-grained candidates (AID), the unweighted mean of the three difficulty levels, 1,100 manually annotated questions on 1,350 real and AI-generated videos from seven generators, at 5 fps frame sampling, each model at its officially recommended decoding (Gemini 3.1 Pro at temperature 1.0 and high thinking level) or greedy, answers parsed by Gemini 3.1 Pro; higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 19 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 9.8 | #99 |
| 2 | Gemini 3 Flash | 9.8 | #175 |
| 3 | Qwen 3 VL 32B (Thinking) | 9.6 | #573 |
| 4 | Qwen 3 VL 30B A3B (Thinking) | 7.8 | #719 |
| 5 | Molmo2-8B | 7.8 | #1621 |
| 6 | VideoVeritas-8B | 7.8 | |
| 7 | Qwen 3 VL 32B Instruct | 7.3 | #606 |
| 8 | Qwen 3 VL 30B A3B Instruct | 6.9 | #895 |
| 9 | MiMo-VL-7B-RL | 6.2 | #1155 |
| 10 | InternVL3.5-30B-A3B | 6.2 | #1267 |
| 11 | Qwen 3 VL 8B (Thinking) | 5.9 | #910 |
| 12 | Skyra-7B-RL | 5.8 | |
| 13 | Skyra-7B-SFT | 5.7 | |
| 14 | Qwen 3 VL 8B Instruct | 5.5 | #1029 |
| 15 | BusterX++ 7B | 5.3 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=artifact-bench-ai-generated-video-artifact-identification · How It Works · Data refreshed daily, snapshot 2026-11-21.