FailSafeQA Context Grounding - Missing Context: leaderboard
The finance question arrives with no document at all (a failed upload) and the model should say so rather than answer from memory or invent figures; the score is the share of such queries it declines or flags. The irrelevant-document slice is easier for almost every model.
Metric: Share of queries without a document declined or flagged (0-1, higher is better). Source: arxiv.org. 23 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 2.5 32B Instruct | 0.68 | #1215 |
| 2 | Gemini 2.0 Flash (Preview) | 0.66 | #736 |
| 3 | Qwen 2.5 7B Instruct | 0.64 | #2564 |
| 4 | Gemini 1.5 Pro (002) | 0.64 | #838 |
| 5 | Llama 3.1 8B Instruct | 0.63 | #4246 |
| 6 | Qwen 2.5 14B Instruct | 0.61 | #1732 |
| 7 | Qwen 2.5 72B Instruct | 0.6 | #1092 |
| 8 | Qwen2.5-7B-Instruct-1M | 0.58 | #2407 |
| 9 | O1 | 0.55 | #393 |
| 10 | Qwen2.5-14B-Instruct-1M | 0.53 | #1414 |
| 11 | O3 Mini | 0.51 | #533 |
| 12 | Llama 3.1 Nemotron 70B Instruct | 0.48 | #1444 |
| 13 | Llama 3.1 70B Instruct | 0.47 | #1546 |
| 14 | GPT-4o | 0.43 | #741 |
| 15 | Llama 3.3 70B Instruct | 0.4 | #1357 |
Interactive version: theaggregate.ai/benchmark?slug=failsafeqa-context-grounding-missing-context · How It Works · Data refreshed daily, snapshot 2026-11-13.