RefereeBench - Foul and Penalty Reasoning: leaderboard

RefereeBench (2026) tests multimodal models as sports referees on 925 officiating videos across 11 sports with 6,475 human-annotated multiple-choice questions. This board is accuracy in percent on questions about why a foul or penalty applies, averaged across sports; higher is better.

Metric: Accuracy (%) on foul and penalty reasoning questions (mean of Q3 and Q5), averaged across 11 sports; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash53.1#175
2Gemini 3 Pro53#145
3Seed 1.850.7#252
4GPT-547.8#178
5Claude Sonnet 4.542.5#261
6GPT-4o39.1#751
7Claude Haiku 4.538.7#583
8InternVL3.5-8B37.1#1604
9Qwen 3 VL 8B37.1#917
10VideoLLaMA3-7B32.3#2050
11LLaVA-Video-7B24.9#2276

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=refereebench-foul-and-penalty-reasoning · How It Works · Data refreshed daily, snapshot 2026-11-21.