Sports Film Review - Frames - svc-bench-3-fresh: leaderboard
Sports Film Review, svc-bench-3 fresh seeds, frames track (mariusCNN/sports-video-coach, 2026): the svc-bench-3 suite drawn anew from an integer seed for each run, "the same four scenes and kinds of traps, with new counts, shot outcomes, release times and stroke rates" (foot touches while juggling, made or missed shots at a far hoop, pass release times, a rower's stroke rate). Each run sees different videos, is scored against its own seed's truth and measured above the blind baseline on that seed. The prepared folder never holds the truth; a seed becomes public once its result file is published. Drawing fresh seeds "reduces the effect of memorising the public answers but proves nothing about training data", as the page puts it. The tool draws each video from numbers, so "the truth is exact. No human labels, no judge model." Unlike multiple-choice video QA, the answers are numbers: counts, rates, event timestamps and made/missed lists, each task scored 0 to 1 against that truth and then measured above a blind-guess floor (a deterministic guesser that sees no video and copies the example answers in the task text): per task above = clip((score - floor) / (1 - floor), 0, 1), and the suite score is the mean over its tasks. Frames track: no video and no code, only stills (one every 0.08 s) and contact sheets of 25 with the time printed under each still; Claude Code runs with --restricted --safe-mode --tools Read, Codex runs read-only with its shell and code-execution features disabled and sees only the contact sheets attached to the prompt (beyond that, its tool restriction is by prompt only). Claude Sonnet 5.5 (Max) is 0 because both of its runs were killed at the timeout (1200 s and 2400 s) before writing an answer. Each row is the source's mean over the runs of one agent, model and settings configuration (1 to 4 runs per row on 2026-10-30); "A run killed at its time limit scores 0". A label suffix (Low, High, Max, xHigh) is the effort the row's settings name; rows without one state the default effort or none. Rows from a local Ollama quantisation are not listed. Higher is better. Source: https://mariuscnn.github.io/sports-video-coach/benchmark/
Metric: Mean score above the blind-guess floor (0-1). Source: mariuscnn.github.io. Saturation forecast: Around February 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra (xHigh) | 1 | #6 (GPT-6 Astra) |
| 2 | Claude Opus 5.5 | 1 | #3 |
| 3 | Claude Sonnet 5.5 (High) | 0.93 | #14 (Claude Sonnet 5.5) |
| 4 | Claude Opus 5.5 (Low) | 0.78 | #3 (Claude Opus 5.5) |
| 5 | Claude Haiku 5.5 (High) | 0.56 | #43 (Claude Haiku 5.5) |
| 6 | Claude Sonnet 5.5 | 0.42 | #14 |
| 7 | Claude Haiku 5.5 | 0.2 | #43 |
| 8 | Claude Sonnet 5.5 (Max) | 0 | #14 (Claude Sonnet 5.5) |
No result here: #7 GPT-6.1 Sol, #9 Gemini 4 Argon, #10 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=sports-film-review-frames-svc-bench-3-fresh · How It Works · Data refreshed daily, snapshot 2026-11-05.