EHRBench - PROMOTE: leaderboard

EHRBench (2026) tests LLMs on clinical decision questions generated from real patient records and verified against biomedical knowledge bases, posed as multiple-choice questions with 4 to 6 options. This board is accuracy in percent on the questions built from PROMOTE records; higher is better.

Metric: Accuracy (%) on the questions built from PROMOTE records, EHR-grounded multiple-choice clinical decision questions (4, 5 and 6 options) built from MIMIC-III, MIMIC-IV and PROMOTE encounter trajectories with knowledge-base verification, JSON-constrained answers, deterministic decoding; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 31 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.2 Instant70.7#309
2GPT-5 Chat68.26#452
3GPT-4.167.87#506
4GLM-4 32B (0414)65.85#1525
5Qwen 3 32B65.55#983
6Llama 3.3 70B Instruct65.35#1394
7GPT-4.1 Mini64.45#799
8GPT-5 Mini63.88#373
9Mistral Small 363.38#1809
10Qwen 2.5 32B63.05#1347
11m1-32B-1k61.46#1247
12EHRBench llama3-70B (checkpoint unspecified)61.2
13doctor-r1-8B59.01#1634
14Qwen 3 8B58.74#1890
15Qwen 3 4B58.39#2379

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=ehrbench-promote · How It Works · Data refreshed daily, snapshot 2026-11-22.