VibeVideoBench Core Spec - All Checks: leaderboard
VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Success requires all program checks to pass across 205 core tasks. The native percentage printed beside the count is retained. Metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra | 92.7 | #3 |
| 2 | Kimi K3 | 83.9 | #34 |
| 3 | GPT-5.6 Sol | 82.4 | #19 |
No result here: #2 Claude Opus 5.5, #6 GPT-6.1 Sol, #8 Gemini 4 Argon.
Interactive version: theaggregate.ai/benchmark?slug=vibevideobench-core-spec-all-checks · How It Works · Data refreshed daily, snapshot 2026-10-28.