MedPhysBench v1.0 - Worksheet False Alarms: leaderboard
Medical physics calculations and detecting planted mistakes under deterministic tolerances. False-alarm rate on correct calculations (%). MedPhysBench frozen v1.0 secret-seed test; single answer, no tool execution,max32768output, provider-default temperature/reasoning; infrastructure requests may retry. Calculation denominator600items except Astra/Fable5.1 prespecified stratified half300; checking150planted+150correct per format except those two75+75. Blocked/interrupted/truncated/unparsable answers remain failures; no-verdict checking response neither false alarm nor acceptance. Native v1.0 includes the paper-disclosed NCRP151 convention/item defects; do not relabel as correctedv1.1 or use parenthesized completion-conditional calculation rates.
Metric: False-alarm rate on correct calculations (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6.1 Sol (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 0 | |
| 2 | Claude Fable 5.1 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=stratified-half) | 0 | |
| 3 | GPT-6 Astra (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=stratified-half) | 1 | |
| 4 | Gemini 4 Argon (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 1 | |
| 5 | Claude Opus 5.5 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 1 | |
| 6 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 2 | |
| 7 | GPT-6 Luna (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 3 | |
| 8 | GPT-OSS-120B (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 5 | |
| 9 | GPT-OSS-20B (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 5 | |
| 10 | DeepSeek V4 Pro (Benchmark Sweep Run: source=MedPhysBenchV1; provider-default temperature/reasoning; max32768output; sample=full) | 42 |
Interactive version: theaggregate.ai/benchmark?slug=medphysbench-v1-0-worksheet-false-alarms · How It Works · Data refreshed daily, snapshot 2026-11-06.