BBC News Video Captioning - Entity Fidelity: leaderboard
Metric: Entity Fidelity Score (0-1, times 100): per-clip harmonic mean of precision and recall of the reference's named entities (persons, locations, organizations, geopolitical entities) found in the caption by fuzzy matching (token ratio at least 85), averaged over clips whose reference names an entity, the authors' BBC News corpus of 9,838 news video clips (at most 300 seconds) with their editorial descriptions as references; each open-source video LLM writes a free-form description of the clip from sampled frames; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemma 3 12B (IT) | 31.4 | #655 |
| 2 | Qwen 2.5 VL 7B Instruct | 19.8 | #643 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=bbc-news-video-captioning-entity-fidelity · How It Works · Data refreshed daily, snapshot 2026-10-11.