SVFSearch - Plan-Act-Replan Agent: leaderboard

A LangGraph plan-act-replan agent choosing image, text, BM25 and multimodal retrieval tools over the frozen offline resources, on SVFSearch (2026), four-choice questions about paused frames of Chinese gaming short videos that need game-specific knowledge, over its 5,000-question test split. Accuracy in percent; higher is better.

Metric: Accuracy (%) on the 5,000-item SVFSearch test split (four-choice questions about a paused Chinese game video frame), given only the frame, question and options, as a LangGraph Plan-Act-Replan agent choosing among image, text, BM25 and multimodal retrieval over the frozen offline resources (top-5, up to six rounds with one tool call each); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Qwen 3.5 9BNon-reasoning79.1#828
2Qwen 3.5 27BNon-reasoning78.6#375
3Qwen 3 VL 32B71.6#622
4Qwen 3 VL 8B63.7#917
5Qwen 2.5 VL 7B59.3#1862

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=svfsearch-plan-act-replan-agent · How It Works · Data refreshed daily, snapshot 2026-11-21.