SCQA (No Search) - Multi-Hop High-Visibility: leaderboard
SCQA, the Supply Chain Query Assessment of the Helicase paper (Long, Zhao, Zheng and Brintrup, University of Cambridge, 2026), asks human-verified questions about the supply chains of consumer, food and electronics or automotive products. This quadrant has 20 list questions that combine several publicly documented facts, answered by direct prompting at temperature 0 without web search. Per-question set F1 between the listed and reference items, from 0 to 1, with items matched by a ChatGPT 5.5 judge; higher is better.
Metric: Set F1 (0-1; 20 multi-hop list questions on publicly documented facts; direct prompting without web search, temperature 0, ChatGPT 5.5 judge). Source: arxiv.org. Saturation forecast: Around July 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 | 0.55 | #100 |
| 2 | DeepSeek V3.2 | 0.48 | #403 |
| 3 | Qwen 3 235B A22B | 0.44 | #622 |
| 4 | GLM-5 | 0.4 | #256 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=scqa-no-search-multi-hop-high-visibility · How It Works · Data refreshed daily, snapshot 2026-11-12.