VEFX-Dataset - Judge Agreement: leaderboard

VEFX-Dataset (2026) is a set of human-annotated instruction-guided video edits scored on instruction following, rendering quality and edit exclusivity. This board is the Spearman correlation, from -1 to 1, between the overall score a vision-language judge gives each edited video and the human overall score on the held-out test split; higher is better.

Metric: Spearman rank correlation (SRCC, -1 to 1) between the judge's overall score for an edited video (mean of its predicted instruction-following, rendering-quality and edit-exclusivity scores under a shared 1-4 rubric prompt) and the human overall score, on the 849-example VEFX-Dataset test split; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3.1 Pro (Preview)0.752#99
2Seed 2.0 Lite0.72#300
3Qwen 3.5 122B A10B0.631#346
4Seed-1.60.63#469
5Qwen 3.5 397B A17B0.601#264
6Gemini 3.1 Flash Lite0.574#371
7EditReward (VEFX-Bench checkpoint unspecified)0.558
8Gemini 2.5 Flash0.383#463

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=vefx-dataset-judge-agreement · How It Works · Data refreshed daily, snapshot 2026-11-21.