SFI-Bench - Troubleshooting: leaderboard

Causal hypothesis and troubleshooting questions about devices seen in the video from SFI-Bench (2026): four-option questions about egocentric videos of scanned indoor rooms, answered without a web search tool. Accuracy in percent; higher is better.

Metric: Accuracy (%) on the causal hypothesis and troubleshooting questions, four-option multiple choice on egocentric indoor videos (ARKitScenes and ScanNet++ scans), zero-shot with the same prompt templates, models at their default configurations; models without a search tool, answered offline; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1LLaVA-OneVision-72B61#1771
2Qwen 3 VL 235B A22B Instruct58.1#532
3InternVL3.5-14B54.7#1504
4Qwen 3 VL 32B Instruct54.6#606
5InternVL3.5-8B52.3#1604
6InternVL3.5-30B-A3B52.3#1267
7LLaVA-Video-72B50.9#1193
8LLaVA-OneVision-7B50.3#2981
9Qwen 3 VL 30B A3B (Thinking)50#719
10Qwen 3 VL 8B Instruct48.8#1029
11Qwen 3 VL 8B (Thinking)47.7#910
12Qwen 3 VL Plus47.7#512
13Qwen 3 VL 30B A3B Instruct45.3#895
14Qwen 3 VL 235B A22B (Thinking)45.3#459
15Qwen 3 VL 32B (Thinking)44.2#573

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sfi-bench-troubleshooting · How It Works · Data refreshed daily, snapshot 2026-11-21.