MR-Bench - Medication Imputation: leaderboard

MR-Bench (2026) tests decision-oriented clinical reasoning on 1,000 MIMIC-IV hospital admissions with eight-option questions, at temperature 0.5. This board asks for the masked medication of a partly shown prescription list, among distractors that include interacting or contraindicated drugs. Accuracy in percent; higher is better.

Metric: Accuracy (%, times 100) at picking the masked medication among distractors that include interacting or contraindicated drugs; 1,000 MIMIC-IV hospital admissions, eight-option questions, temperature 0.5, an answer counted from the first or the last option letter the output names, whichever scores higher over the dataset; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 24 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3 Pro55#145
2GPT-554.4#178
3DeepSeek V3.2Thinking48.4#419
4Qwen 3 Max44.6#415
5GPT-4o36.3#751
6MedS3-8B28.7
7Qwen 3 4B 2507 (Thinking)27.2#1433
8Qwen 3 8B26.9#1873
9Med42-v2-8B24.9#2558
10BioMed-R1-8B24.2
11Gemma 3 4B (IT)23.7#4139
12Llama 3.1 8B UltraMedical23.1
13Adathink-Med-Llama-8B22.6
14HuatuoGPT-o1-8B22.5#2983
15ClinicalGPT-R1-Qwen-7B-EN-preview22.5

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mr-bench-medication-imputation · How It Works · Data refreshed daily, snapshot 2026-11-21.