video-SALMONN-2 Test Set (GPT-4.1 judge) - Total error: leaderboard
AVoCaDO paper Table 1, the producing source of the baseline rows later re-reported by DiaDem and EVA-Cap: video-SALMONN-2 caption test set scored with GPT-4.1 as the judge (replacing the original GPT-3.5 judge); total error is the missing-event rate plus the hallucination rate, so lower is better and it can exceed 100. Results reproduced with each benchmark's official code. InternVL3.5-8B and Qwen2.5-VL-7B receive the visual stream only (no audio); every other model receives audio and video. AVoCaDO is the paper's own 7B captioner. Metric: Missed + hallucinated events (%). Lower is better. Source: https://arxiv.org/html/2510.10395v1
Metric: Missed + hallucinated events (%). Source: arxiv.org. Saturation forecast: Around November 2026. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 31.3 | #151 |
| 2 | Gemini 2.5 Flash | 33.3 | #242 |
| 3 | Qwen3 Omni 30B A3B Instruct | 45.6 | #379 |
| 4 | Qwen2.5-Omni-7B | 57.1 | #622 |
| 5 | Qwen 2.5 VL 7B | 57.5 | #642 |
No result here: #4 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=video-salmonn-2-test-set-gpt-4-1-judge-total-error · How It Works · Data refreshed daily, snapshot 2026-10-18.