FailSafeQA Context Grounding: leaderboard
The model gets a finance question about a 10-K filing whose document is missing or is an unrelated filing, and should say it cannot answer instead of inventing one; the score is the share of such cases handled that way, averaged over both. Unlike the paper's Robustness score, which grades answers to answerable questions, this counts only declining or flagging.
Metric: Context Grounding: share of unanswerable queries declined or flagged (0-1, higher is better). Source: arxiv.org. 23 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 2.5 32B Instruct | 0.79 | #1215 |
| 2 | Gemini 2.0 Flash (Preview) | 0.74 | #736 |
| 3 | Qwen 2.5 7B Instruct | 0.7 | #2564 |
| 4 | Gemini 1.5 Pro (002) | 0.69 | #838 |
| 5 | Qwen 2.5 14B Instruct | 0.68 | #1732 |
| 6 | Llama 3.1 8B Instruct | 0.65 | #4246 |
| 7 | Qwen2.5-14B-Instruct-1M | 0.65 | #1414 |
| 8 | Qwen 2.5 72B Instruct | 0.64 | #1092 |
| 9 | Qwen2.5-7B-Instruct-1M | 0.6 | #2407 |
| 10 | O3 Mini | 0.59 | #533 |
| 11 | O1 | 0.55 | #393 |
| 12 | Llama 3.1 Nemotron 70B Instruct | 0.5 | #1444 |
| 13 | GPT-4o | 0.47 | #741 |
| 14 | Llama 3.1 70B Instruct | 0.47 | #1546 |
| 15 | Llama 3.3 70B Instruct | 0.45 | #1357 |
Interactive version: theaggregate.ai/benchmark?slug=failsafeqa-context-grounding · How It Works · Data refreshed daily, snapshot 2026-11-13.