Sports Film Review - Frames - svc-bench-2: leaderboard

Sports Film Review, svc-bench-2 (hard), frames track (mariusCNN/sports-video-coach, 2026): a model reviews five synthetic sports videos (28 to 56 s) by eye, with a hand-held camera, look-alike decoys, rule-defined events and sensor noise: one player's dribbles (a second dribbler has an identical ball, and the ball passes behind a pillar), good squat reps by a written rule (a double dip is one rep, a squat that never locks out does not count), foot strikes per minute from 20 s to the end, the times a runner arrives at a cone where he stops or turns back, and made or missed for each shot (a pump fake is not a shot; a ball that dips in and pops out is a miss). The tool draws each video from numbers, so "the truth is exact. No human labels, no judge model." Unlike multiple-choice video QA, the answers are numbers: counts, rates, event timestamps and made/missed lists, each task scored 0 to 1 against that truth and then measured above a blind-guess floor (a deterministic guesser that sees no video and copies the example answers in the task text): per task above = clip((score - floor) / (1 - floor), 0, 1), and the suite score is the mean over its tasks. Frames track: no video and no code, only stills (one every 0.08 s) and contact sheets of 25 with the time printed under each still; Claude Code runs with --restricted --safe-mode --tools Read, Codex runs read-only with its shell and code-execution features disabled and sees only the contact sheets attached to the prompt (beyond that, its tool restriction is by prompt only). The suite is fixed: every draw is byte-identical, and its truth ships in the public package. The author built svc-bench-3 after strong models scored 1.0 here by eye. Each row is the source's mean over the runs of one agent, model and settings configuration (1 to 2 runs per row on 2026-10-30); "A run killed at its time limit scores 0". A label suffix (Low, High, Max, xHigh) is the effort the row's settings name; rows without one state the default effort or none. Rows from a local Ollama quantisation are not listed. Higher is better. Source: https://mariuscnn.github.io/sports-video-coach/benchmark/

Metric: Mean score above the blind-guess floor (0-1). Source: mariuscnn.github.io. Saturation forecast: Around February 2027. 3 models tracked.

Top models

No result here: #7 GPT-6.1 Sol, #9 Gemini 4 Argon, #10 Claude Fable 5.1.

Interactive version: theaggregate.ai/benchmark?slug=sports-film-review-frames-svc-bench-2 · How It Works · Data refreshed daily, snapshot 2026-11-05.