SFI-Bench - Operation Planning: leaderboard

Operation planning questions, on how to operate appliances and devices seen in the video, from SFI-Bench (2026): four-option questions about egocentric videos of scanned indoor rooms, answered without a web search tool. Accuracy in percent; higher is better.

Metric: Accuracy (%) on the operation planning questions, four-option multiple choice on egocentric indoor videos (ARKitScenes and ScanNet++ scans), zero-shot with the same prompt templates, models at their default configurations; models without a search tool, answered offline; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1LLaVA-OneVision-72B58.4#1771
2LLaVA-Video-72B58.4#1193
3Qwen 3 VL 8B Instruct54.9#1029
4Qwen 3 VL 32B Instruct53.1#606
5InternVL3.5-8B53.1#1604
6Qwen 3 VL 235B A22B Instruct53#532
7InternVL3.5-14B52.2#1504
8Qwen 3 VL 235B A22B (Thinking)51.3#459
9InternVL3.5-30B-A3B51.3#1267
10Qwen 3 VL Plus50.4#512
11LLaVA-OneVision-7B49.9#2981
12Qwen 3 VL 30B A3B Instruct49.6#895
13Qwen 3 VL 32B (Thinking)42.5#573
14GLM-4.5VThinking42.5#754
15Qwen 3 VL 8B (Thinking)40.7#910

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sfi-bench-operation-planning · How It Works · Data refreshed daily, snapshot 2026-11-21.