MedPRMBench (Step Probability): leaderboard

Step-level error detection in medical reasoning chains on MedPRMBench (2026), process reward benchmark chains with 14 types of injected and expert-revised reasoning errors, each step judged from the probability the model gives to the correct token over the erroneous token. PRMScore, the mean of the F1 for erroneous steps and the F1 for correct steps, in percent; higher is better.

Metric: PRMScore (%): mean of the F1 for erroneous steps and the F1 for correct steps (weights 0.5 and 0.5) in step-level error detection over the erroneous reasoning chains of the 6,500 MedPRMBench test instances (14 injected medical reasoning error types), step-probability protocol: at the delimiter after each step the model's probability of the plus token against the minus token is read, and a step is correct when that probability is at least 0.5; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 12 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Sky-T1-32B-Flash65.4#1366
2txgemma-9B-chat57.7#8041
3Llama 3.1 8B UltraMedical56.6
4Meerkat-8B55.3#1997
5MedSSS_Policy54.5
6Marco-o154.3#2811
7QwQ-32B52.4#907
8HuatuoGPT-o1-8B51.8#2983
9DeepSeek R1 Distill Llama 70B51.1#1669
10Qwen 3 8B51#1873
11DeepSeek R1 Distill Qwen 32B49.7#1834
12Meditron3-8B46.8

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=medprmbench-step-probability · How It Works · Data refreshed daily, snapshot 2026-11-21.