HalluCXR: leaderboard
HalluCXR (2026) measures how often vision-language models hallucinate when reading chest radiographs: 856 MIMIC-CXR images stratified from normal to complex, each queried for a report, a targeted yes/no finding and a differential diagnosis. The score is the share (%) of responses with a fabricated or omitted finding against CheXpert labels, by negation-aware keyword detection; lower is better.
Metric: Hallucination rate (%): share of responses with at least one fabricated or omitted CheXpert finding by negation-aware keyword detection, over 856 MIMIC-CXR radiographs and three query types (report, targeted yes/no question, differential diagnosis), temperature 0; lower is better. Source: arxiv.org. Saturation forecast: Around August 2027. 5 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | LLaVA 1.6 Mistral 7B [4bit] | 67.1 | |
| 2 | GPT-5.1 | 71.1 | #240 |
| 3 | Gemini 3 Flash | 71.2 | #175 |
| 4 | HalluCXR InternVL2 6B [4bit] (checkpoint unspecified) | 76.6 | |
| 5 | Qwen2-VL-7B [4bit] | 82.3 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=hallucxr · How It Works · Data refreshed daily, snapshot 2026-11-21.