Artifact-Bench (AI-Generated Video) - Pairwise Realism Comparison: leaderboard
Artifact-Bench (2026) tests whether multimodal LLMs perceive the artifacts of AI-generated videos, on 1,100 manually annotated questions over real videos and clips from seven video generators, split into three difficulty levels. This task asks which of two AI-generated videos is more realistic; accuracy is the mean of the three difficulty levels, in percent, higher is better.
Metric: Accuracy (%) on the 250 questions asking which of two AI-generated videos is more realistic (PVRC), the unweighted mean of the three difficulty levels, 1,100 manually annotated questions on 1,350 real and AI-generated videos from seven generators, at 5 fps frame sampling, each model at its officially recommended decoding (Gemini 3.1 Pro at temperature 1.0 and high thinking level) or greedy, answers parsed by Gemini 3.1 Pro; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 19 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | VideoVeritas-8B | 53.1 | |
| 2 | Gemini 3 Flash | 50.9 | #180 |
| 3 | Qwen 3 VL 32B Instruct | 50.8 | #607 |
| 4 | MiMo-VL-7B-SFT | 49.8 | #1097 |
| 5 | Gemini 3.1 Pro (Preview) | 48.6 | #102 |
| 6 | MiMo-VL-7B-RL | 46.9 | #1160 |
| 7 | InternVL3.5-8B | 46.5 | #1618 |
| 8 | Qwen 3 VL 8B Instruct | 45.8 | #1043 |
| 9 | Qwen 3 VL 30B A3B (Thinking) | 44.9 | #727 |
| 10 | Keye-VL-1.5-8B | 44.7 | #1729 |
| 11 | Qwen 3 VL 30B A3B Instruct | 44.6 | #903 |
| 12 | Qwen 3 VL 32B (Thinking) | 43.8 | #577 |
| 13 | InternVL3.5-38B | 42.9 | #1026 |
| 14 | BusterX++ 7B | 42.5 | |
| 15 | Molmo2-8B | 40 | #1628 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=artifact-bench-ai-generated-video-pairwise-realism-comparison · How It Works · Data refreshed daily, snapshot 2026-11-22.