SCQA (No Search) - Multi-Hop Low-Visibility: leaderboard
SCQA, the Supply Chain Query Assessment of the Helicase paper (Long, Zhao, Zheng and Brintrup, University of Cambridge, 2026), asks human-verified questions about the supply chains of consumer, food and electronics or automotive products. This quadrant has 20 questions that require assembling a supply-chain subgraph from fragmented low-visibility sources, answered by direct prompting at temperature 0 without web search. Graph F1 from 0 to 1 (0.6 entity F1 plus 0.4 relation F1 against a reference graph), with entities and relations matched by a ChatGPT 5.5 judge; higher is better.
Metric: Graph F1 (0-1; 0.6 x entity F1 + 0.4 x relation F1 against a reference supply-chain graph; 20 multi-hop questions on facts found only in niche sources; direct prompting without web search, temperature 0, ChatGPT 5.5 judge). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 | 0.63 | #100 |
| 2 | GLM-5 | 0.33 | #256 |
| 3 | DeepSeek V3.2 | 0.27 | #403 |
| 4 | Qwen 3 235B A22B | 0.25 | #622 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=scqa-no-search-multi-hop-low-visibility · How It Works · Data refreshed daily, snapshot 2026-11-12.