VEFX-Dataset - Judge Agreement: leaderboard
VEFX-Dataset (2026) is a set of human-annotated instruction-guided video edits scored on instruction following, rendering quality and edit exclusivity. This board is the Spearman correlation, from -1 to 1, between the overall score a vision-language judge gives each edited video and the human overall score on the held-out test split; higher is better.
Metric: Spearman rank correlation (SRCC, -1 to 1) between the judge's overall score for an edited video (mean of its predicted instruction-following, rendering-quality and edit-exclusivity scores under a shared 1-4 rubric prompt) and the human overall score, on the 849-example VEFX-Dataset test split; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 0.752 | #99 |
| 2 | Seed 2.0 Lite | 0.72 | #300 |
| 3 | Qwen 3.5 122B A10B | 0.631 | #346 |
| 4 | Seed-1.6 | 0.63 | #469 |
| 5 | Qwen 3.5 397B A17B | 0.601 | #264 |
| 6 | Gemini 3.1 Flash Lite | 0.574 | #371 |
| 7 | EditReward (VEFX-Bench checkpoint unspecified) | 0.558 | |
| 8 | Gemini 2.5 Flash | 0.383 | #463 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=vefx-dataset-judge-agreement · How It Works · Data refreshed daily, snapshot 2026-11-21.