SVFSearch - Plan-Act-Replan Agent: leaderboard
A LangGraph plan-act-replan agent choosing image, text, BM25 and multimodal retrieval tools over the frozen offline resources, on SVFSearch (2026), four-choice questions about paused frames of Chinese gaming short videos that need game-specific knowledge, over its 5,000-question test split. Accuracy in percent; higher is better.
Metric: Accuracy (%) on the 5,000-item SVFSearch test split (four-choice questions about a paused Chinese game video frame), given only the frame, question and options, as a LangGraph Plan-Act-Replan agent choosing among image, text, BM25 and multimodal retrieval over the frozen offline resources (top-5, up to six rounds with one tool call each); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Qwen 3.5 9B | Non-reasoning | 79.1 | #828 |
| 2 | Qwen 3.5 27B | Non-reasoning | 78.6 | #375 |
| 3 | Qwen 3 VL 32B | 71.6 | #622 | |
| 4 | Qwen 3 VL 8B | 63.7 | #917 | |
| 5 | Qwen 2.5 VL 7B | 59.3 | #1862 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=svfsearch-plan-act-replan-agent · How It Works · Data refreshed daily, snapshot 2026-11-21.