SVFSearch - Direct QA: leaderboard

Direct answering, from the frame, question and options without retrieval, on SVFSearch (2026), four-choice questions about paused frames of Chinese gaming short videos that need game-specific knowledge, over its 5,000-question test split. Accuracy in percent; higher is better.

Metric: Accuracy (%) on the 5,000-item SVFSearch test split (four-choice questions about a paused Chinese game video frame), given only the frame, question and options, answered directly without retrieval tools; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3.1 Pro (Preview)77.5#102
2Claude Opus 4.769#86
3GPT-5.467.9#138
4Qwen 3.5 27BNon-reasoning66.4#380
5Qwen 3 VL 32B (Thinking)62.2#577
6Qwen 3.5 9BNon-reasoning59.9#838
7Qwen 3 VL 32B57.1#624
8Qwen 3 VL 8B54.8#927
9Qwen 3 VL 8B (Thinking)53.7#924
10Qwen 2.5 VL 7B49.8#1878

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=svfsearch-direct-qa · How It Works · Data refreshed daily, snapshot 2026-11-22.