EHRNote-ChatQA - Evidence Grounding: leaderboard
EHRNote-ChatQA (2026) poses multi-turn 5-way multiple-choice questions over the full sequence of MIMIC-IV discharge summaries of one patient, pairing each content question with an evidence-grounding question about which notes and sections support the answer. This board is evidence-grounding accuracy in percent over all turns; higher is better.
Metric: QA-level evidence-grounding accuracy (%): choosing the minimal set of notes and section headers that supports the preceding content answer, micro-averaged over all turns, on 967 multi-turn samples, each over the full set of MIMIC-IV discharge summaries of one patient (1 to 5 notes), 8,036 content and 8,036 paired evidence-grounding 5-way multiple-choice questions (chance 20), chat history kept across turns, greedy decoding where settable; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 22 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 90.1 | #144 |
| 2 | GPT-5.4 | 89.02 | #123 |
| 3 | Qwen 3 Next 80B A3B Instruct | 82.09 | #766 |
| 4 | Llama 4 Scout Instruct | 76 | #1543 |
| 5 | Qwen 3 30B A3B 2507 Instruct | 74.9 | #1089 |
| 6 | GPT-5.4 Mini | 74.05 | #389 |
| 7 | DeepSeek R1 Distill Llama 70B | 65.64 | #1637 |
| 8 | Qwen 3 4B 2507 Instruct | 62.82 | #2163 |
| 9 | DeepSeek R1 Distill Qwen 32B | 58.66 | #1788 |
| 10 | Ministral-3-3B-Instruct-2512 | 55.46 | #2579 |
| 11 | DeepSeek R1 Distill Qwen 14B | 53.94 | #2582 |
| 12 | Ministral-3-8B-Instruct-2512 | 50.86 | #1858 |
| 13 | Phi-4 Mini Instruct | 48.83 | #6419 |
| 14 | MedGemma-4B-IT | 44.26 | #2716 |
| 15 | Ministral-3-14B-Instruct-2512 | 42.46 | #1593 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=ehrnote-chatqa-evidence-grounding · How It Works · Data refreshed daily, snapshot 2026-11-13.