SYNCR - Numerical Comparison: leaderboard

Accuracy (%) on comparing counts of events across videos in SYNCR (2026), a synthetic benchmark of multi-video questions with programmatically verified answers; zero-shot four-option multiple choice, exact match, 200 questions per task (chance 25). Higher is better.

Metric: Exact-match accuracy (%) on the SYNCR Numerical Comparison task (comparing counts of events across videos), zero-shot four-option multiple choice over 2 to 4 labelled input videos, exact match, deterministic decoding, on the paper's balanced subset of 200 questions per task; chance is 25; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 18 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.460#135
2Qwen 3 VL 32B Instruct44#606
3Qwen 3 VL 8B Instruct33.5#1029
4Qwen 2.5 VL 72B Instruct32.5#841
5Gemini 3 Flash32#175
6Qwen 2.5 VL 32B Instruct30.5#1062
7InternVL3.5-4B28.5#1930
8LLaVA-OV-72B28#1879
9InternVL3.5-14B26.5#1504
10Qwen 3 VL 2B Instruct24.5#2367
11LLaVA-OV-7B22.5#4475
12Qwen 3 VL 4B Instruct21#1332
13LLaVA-OV-0.5B21#4352
14InternVL3.5-1B19#6112
15Qwen 2.5 VL 7B Instruct17.5#1902

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=syncr-numerical-comparison · How It Works · Data refreshed daily, snapshot 2026-11-21.