SVFSearch - Direct QA: leaderboard
Direct answering, from the frame, question and options without retrieval, on SVFSearch (2026), four-choice questions about paused frames of Chinese gaming short videos that need game-specific knowledge, over its 5,000-question test split. Accuracy in percent; higher is better.
Metric: Accuracy (%) on the 5,000-item SVFSearch test split (four-choice questions about a paused Chinese game video frame), given only the frame, question and options, answered directly without retrieval tools; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 77.5 | #102 | |
| 2 | Claude Opus 4.7 | 69 | #86 | |
| 3 | GPT-5.4 | 67.9 | #138 | |
| 4 | Qwen 3.5 27B | Non-reasoning | 66.4 | #380 |
| 5 | Qwen 3 VL 32B (Thinking) | 62.2 | #577 | |
| 6 | Qwen 3.5 9B | Non-reasoning | 59.9 | #838 |
| 7 | Qwen 3 VL 32B | 57.1 | #624 | |
| 8 | Qwen 3 VL 8B | 54.8 | #927 | |
| 9 | Qwen 3 VL 8B (Thinking) | 53.7 | #924 | |
| 10 | Qwen 2.5 VL 7B | 49.8 | #1878 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=svfsearch-direct-qa · How It Works · Data refreshed daily, snapshot 2026-11-22.