MR-Bench - Procedure Selection: leaderboard

MR-Bench (2026) tests decision-oriented clinical reasoning on 1,000 MIMIC-IV hospital admissions with eight-option questions, at temperature 0.5. This board asks for the set of procedures performed in the admission, among procedures of other patients. Accuracy in percent; higher is better.

Metric: Accuracy (%, times 100) at picking the admission's procedure set against other patients' procedures; 1,000 MIMIC-IV hospital admissions, eight-option questions, temperature 0.5, an answer counted from the first or the last option letter the output names, whichever scores higher over the dataset; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 24 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-573.8#178
2DeepSeek V3.2Thinking69.4#419
3Qwen 3 Max68.2#415
4Gemini 3 Pro67.6#145
5GPT-4o64.9#751
6Qwen 3 8B60.2#1873
7HuatuoGPT-o1-7B59.9#2282
8Qwen 2.5 7B Instruct59.5#2723
9Qwen 3 4B 2507 (Thinking)58.8#1433
10Llama 3.1 8B UltraMedical58.6
11Med42-v2-8B58.5#2558
12MedS3-8B57.8
13m1-7B-23K57.5#5309
14Adathink-Med-Llama-8B57.4
15BioMed-R1-8B55.2

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mr-bench-procedure-selection · How It Works · Data refreshed daily, snapshot 2026-11-21.