VibeVideoBench Core Spec - Editable Project: leaderboard
VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. There are 205 core tasks: 54 require an exact frame count, and 98 require audio with a peak above -50 dBFS. Missing videos fail. Editable Project. Metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra | 96.1 | #3 |
| 2 | Kimi K3 | 87.3 | #34 |
| 3 | GPT-5.6 Sol | 86.8 | #19 |
No result here: #2 Claude Opus 5.5, #6 GPT-6.1 Sol, #8 Gemini 4 Argon.
Interactive version: theaggregate.ai/benchmark?slug=vibevideobench-core-spec-editable-project · How It Works · Data refreshed daily, snapshot 2026-10-28.