MedRedFlag - False Assumptions Accommodated: leaderboard
MedRedFlag (Sambara et al., Duke University, 2026) uses real r/AskDocs patient questions in which a verified clinician redirected the patient because the question rested on a false or unsafe assumption, such as asking how to remove a splinter at home when the wound needs emergency care. The score is the percentage of 100 test questions whose response still provides the requested information, which implicitly treats the false assumption as valid, judged by GPT-5 against a physician-written criterion for each question; lower is better.
Metric: Share of 100 questions (%) whose response still provides the requested information, implicitly treating the false assumption as valid (GPT-5 judge against a physician-written criterion; lower is better). Source: arxiv.org. Saturation forecast: Around June 2028. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.5 | 60 | #152 |
| 2 | GPT-5 | 73 | #163 |
| 3 | Llama 3.3 70B Instruct | 74 | #1353 |
| 4 | Qwen 3 32B | 80 | #960 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=medredflag-false-assumptions-accommodated · How It Works · Data refreshed daily, snapshot 2026-11-12.