MedPRMBench (Step Probability): leaderboard
Step-level error detection in medical reasoning chains on MedPRMBench (2026), process reward benchmark chains with 14 types of injected and expert-revised reasoning errors, each step judged from the probability the model gives to the correct token over the erroneous token. PRMScore, the mean of the F1 for erroneous steps and the F1 for correct steps, in percent; higher is better.
Metric: PRMScore (%): mean of the F1 for erroneous steps and the F1 for correct steps (weights 0.5 and 0.5) in step-level error detection over the erroneous reasoning chains of the 6,500 MedPRMBench test instances (14 injected medical reasoning error types), step-probability protocol: at the delimiter after each step the model's probability of the plus token against the minus token is read, and a step is correct when that probability is at least 0.5; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 12 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Sky-T1-32B-Flash | 65.4 | #1366 |
| 2 | txgemma-9B-chat | 57.7 | #8041 |
| 3 | Llama 3.1 8B UltraMedical | 56.6 | |
| 4 | Meerkat-8B | 55.3 | #1997 |
| 5 | MedSSS_Policy | 54.5 | |
| 6 | Marco-o1 | 54.3 | #2811 |
| 7 | QwQ-32B | 52.4 | #907 |
| 8 | HuatuoGPT-o1-8B | 51.8 | #2983 |
| 9 | DeepSeek R1 Distill Llama 70B | 51.1 | #1669 |
| 10 | Qwen 3 8B | 51 | #1873 |
| 11 | DeepSeek R1 Distill Qwen 32B | 49.7 | #1834 |
| 12 | Meditron3-8B | 46.8 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=medprmbench-step-probability · How It Works · Data refreshed daily, snapshot 2026-11-21.