SYNCR - Numerical Comparison: leaderboard
Accuracy (%) on comparing counts of events across videos in SYNCR (2026), a synthetic benchmark of multi-video questions with programmatically verified answers; zero-shot four-option multiple choice, exact match, 200 questions per task (chance 25). Higher is better.
Metric: Exact-match accuracy (%) on the SYNCR Numerical Comparison task (comparing counts of events across videos), zero-shot four-option multiple choice over 2 to 4 labelled input videos, exact match, deterministic decoding, on the paper's balanced subset of 200 questions per task; chance is 25; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 18 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 60 | #135 |
| 2 | Qwen 3 VL 32B Instruct | 44 | #606 |
| 3 | Qwen 3 VL 8B Instruct | 33.5 | #1029 |
| 4 | Qwen 2.5 VL 72B Instruct | 32.5 | #841 |
| 5 | Gemini 3 Flash | 32 | #175 |
| 6 | Qwen 2.5 VL 32B Instruct | 30.5 | #1062 |
| 7 | InternVL3.5-4B | 28.5 | #1930 |
| 8 | LLaVA-OV-72B | 28 | #1879 |
| 9 | InternVL3.5-14B | 26.5 | #1504 |
| 10 | Qwen 3 VL 2B Instruct | 24.5 | #2367 |
| 11 | LLaVA-OV-7B | 22.5 | #4475 |
| 12 | Qwen 3 VL 4B Instruct | 21 | #1332 |
| 13 | LLaVA-OV-0.5B | 21 | #4352 |
| 14 | InternVL3.5-1B | 19 | #6112 |
| 15 | Qwen 2.5 VL 7B Instruct | 17.5 | #1902 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=syncr-numerical-comparison · How It Works · Data refreshed daily, snapshot 2026-11-21.