HalluCXR: leaderboard

HalluCXR (2026) measures how often vision-language models hallucinate when reading chest radiographs: 856 MIMIC-CXR images stratified from normal to complex, each queried for a report, a targeted yes/no finding and a differential diagnosis. The score is the share (%) of responses with a fabricated or omitted finding against CheXpert labels, by negation-aware keyword detection; lower is better.

Metric: Hallucination rate (%): share of responses with at least one fabricated or omitted CheXpert finding by negation-aware keyword detection, over 856 MIMIC-CXR radiographs and three query types (report, targeted yes/no question, differential diagnosis), temperature 0; lower is better. Source: arxiv.org. Saturation forecast: Around August 2027. 5 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1LLaVA 1.6 Mistral 7B [4bit]67.1
2GPT-5.171.1#240
3Gemini 3 Flash71.2#175
4HalluCXR InternVL2 6B [4bit] (checkpoint unspecified)76.6
5Qwen2-VL-7B [4bit]82.3

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=hallucxr · How It Works · Data refreshed daily, snapshot 2026-11-21.