MamaBench - Base Accuracy: leaderboard
Metric: Base accuracy (%; share of pairs whose base-case diagnosis is correct; 217 expert-authored maternal and paediatric patient narratives, each paired with a minimally perturbed counterfactual whose diagnosis differs (371 pathologies); diagnoses judged by Claude Opus 4.5 at temperature 0; standalone model at temperature 0, no retrieval). Source: arxiv.org. Saturation forecast: Around December 2026. 5 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Sonnet 4.6 | 82 |
| 2 | GPT-5.4 | 77.4 |
| 3 | Kimi K2.5 | 76 |
| 4 | Nemotron 3 Super | 72.4 |
| 5 | GLM-5 | 54.8 |
Interactive version: theaggregate.ai/benchmark?slug=mamabench-base-accuracy · How It Works · Data refreshed daily, snapshot 2026-09-29.