MR-Bench: leaderboard

MR-Bench (2026) tests decision-oriented clinical reasoning on 1,000 MIMIC-IV hospital admissions with eight-option questions, at temperature 0.5. This board is the mean accuracy of its medication-imputation and procedure-selection tasks, in percent; higher is better.

Metric: Accuracy (%, times 100), mean of the medication-imputation and procedure-selection tasks; 1,000 MIMIC-IV hospital admissions, eight-option questions, temperature 0.5, an answer counted from the first or the last option letter the output names, whichever scores higher over the dataset; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 24 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-564.1#178
2Gemini 3 Pro61.3#145
3DeepSeek V3.2Thinking58.9#419
4Qwen 3 Max56.4#415
5GPT-4o50.6#751
6Qwen 3 8B43.6#1873
7MedS3-8B43.3
8Qwen 3 4B 2507 (Thinking)43#1433
9Med42-v2-8B41.7#2558
10HuatuoGPT-o1-7B41#2282
11Llama 3.1 8B UltraMedical40.9
12Qwen 2.5 7B Instruct40.4#2723
13Adathink-Med-Llama-8B40
14BioMed-R1-8B39.7
15m1-7B-23K39.1#5309

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mr-bench · How It Works · Data refreshed daily, snapshot 2026-11-21.