MedPhysBench v1.0 - Calculation: leaderboard
Medical physics calculations and detecting planted mistakes under deterministic tolerances. Within-tolerance calculation accuracy (%). MedPhysBench frozen v1.0 secret-seed test; single answer, no tool execution,max32768output, provider-default temperature/reasoning; infrastructure requests may retry. Calculation denominator600items except Astra/Fable5.1 prespecified stratified half300; checking150planted+150correct per format except those two75+75. Blocked/interrupted/truncated/unparsable answers remain failures; no-verdict checking response neither false alarm nor acceptance. Native v1.0 includes the paper-disclosed NCRP151 convention/item defects; do not relabel as correctedv1.1 or use parenthesized completion-conditional calculation rates.
Metric: Within-tolerance calculation accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=stratified-half) | 100 | |
| 2 | GPT-6.1 Sol (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 99.7 | |
| 3 | Gemini 4 Argon (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 99.5 | |
| 4 | Claude Opus 5.5 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 99.3 | |
| 5 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 93.7 | |
| 6 | GPT-6 Luna (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 93.5 | |
| 7 | Claude Fable 5.1 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=stratified-half) | 90 | |
| 8 | DeepSeek V4 Pro (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 89 | |
| 9 | GPT-OSS-120B (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 85.7 | |
| 10 | Mistral Large 3 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 84.8 |
Interactive version: theaggregate.ai/benchmark?slug=medphysbench-v1-0-calculation · How It Works · Data refreshed daily, snapshot 2026-11-06.