SFI-Bench - Operation Planning (Web Search): leaderboard

Operation planning questions, on how to operate appliances and devices seen in the video, from SFI-Bench (2026): four-option questions about egocentric videos of scanned indoor rooms, answered by models that may first retrieve external knowledge such as user manuals with a web search tool. Accuracy in percent; higher is better.

Metric: Accuracy (%) on the operation planning questions, four-option multiple choice on egocentric indoor videos (ARKitScenes and ScanNet++ scans), zero-shot with the same prompt templates, models at their default configurations; the model may first retrieve external knowledge such as user manuals through its web search tool; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3.1 Flash Lite77.7#377
2Gemini 3.1 Pro (Preview)67.9#102
3GPT-5.4High65.5#138
4O4 Mini65#372
5Gemini 2.5 Pro60.2#280
6GPT-560.2#185
7Gemini 2.5 Flash50.4#466

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sfi-bench-operation-planning-web-search · How It Works · Data refreshed daily, snapshot 2026-11-22.