SFI-Bench - Troubleshooting (Web Search): leaderboard
Causal hypothesis and troubleshooting questions about devices seen in the video from SFI-Bench (2026): four-option questions about egocentric videos of scanned indoor rooms, answered by models that may first retrieve external knowledge such as user manuals with a web search tool. Accuracy in percent; higher is better.
Metric: Accuracy (%) on the causal hypothesis and troubleshooting questions, four-option multiple choice on egocentric indoor videos (ARKitScenes and ScanNet++ scans), zero-shot with the same prompt templates, models at their default configurations; the model may first retrieve external knowledge such as user manuals through its web search tool; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 72.1 | #102 | |
| 2 | GPT-5.4 | High | 68.8 | #138 |
| 3 | Gemini 3.1 Flash Lite | 66.3 | #377 | |
| 4 | O4 Mini | 60.4 | #372 | |
| 5 | Gemini 2.5 Pro | 58.1 | #280 | |
| 6 | GPT-5 | 58.1 | #185 | |
| 7 | Gemini 2.5 Flash | 47.7 | #466 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=sfi-bench-troubleshooting-web-search · How It Works · Data refreshed daily, snapshot 2026-11-22.