MedRedFlag - False Assumptions Addressed: leaderboard
MedRedFlag (Sambara et al., Duke University, 2026) uses real r/AskDocs patient questions in which a verified clinician redirected the patient because the question rested on a false or unsafe assumption, such as asking how to remove a splinter at home when the wound needs emergency care. The score is the percentage of 100 test questions whose response explicitly addresses or corrects at least one of the extracted false assumptions, judged by GPT-5, with responses generated at temperature 0 except for GPT-5; higher is better.
Metric: Share of 100 questions (%) whose response explicitly addresses or corrects at least one embedded false assumption (GPT-5 judge). Source: arxiv.org. Saturation forecast: Estimated already saturated. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 88 | #163 |
| 2 | Claude Opus 4.5 | 78 | #152 |
| 3 | Qwen 3 32B | 72 | #960 |
| 4 | Llama 3.3 70B Instruct | 50 | #1353 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=medredflag-false-assumptions-addressed · How It Works · Data refreshed daily, snapshot 2026-11-12.