MedPhysBench v1.0 - Result Detection: leaderboard
Medical physics calculations and detecting planted mistakes under deterministic tolerances. Planted-error detection rate (%). MedPhysBench frozen v1.0 secret-seed test; single answer, no tool execution,max32768output, provider-default temperature/reasoning; infrastructure requests may retry. Calculation denominator600items except Astra/Fable5.1 prespecified stratified half300; checking150planted+150correct per format except those two75+75. Blocked/interrupted/truncated/unparsable answers remain failures; no-verdict checking response neither false alarm nor acceptance. Native v1.0 includes the paper-disclosed NCRP151 convention/item defects; do not relabel as correctedv1.1 or use parenthesized completion-conditional calculation rates.
Metric: Planted-error detection rate (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 4 Argon (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 100 | |
| 2 | Claude Opus 5.5 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 99 | |
| 3 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 99 | |
| 4 | GPT-6 Astra (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=stratified-half) | 97 | |
| 5 | GPT-6.1 Sol (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 97 | |
| 6 | Claude Fable 5.1 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=stratified-half) | 97 | |
| 7 | GPT-OSS-120B (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 97 | |
| 8 | GPT-OSS-20B (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 96 | |
| 9 | GPT-6 Luna (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 95 | |
| 10 | Gemma 3 27B (IT) (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 95 |
Interactive version: theaggregate.ai/benchmark?slug=medphysbench-v1-0-result-detection · How It Works · Data refreshed daily, snapshot 2026-11-06.