MamaBench - Robust Accuracy: leaderboard
Metric: Robust accuracy (%; share of pairs with both the base and the counterfactual diagnosis correct; 217 expert-authored maternal and paediatric patient narratives, each paired with a minimally perturbed counterfactual whose diagnosis differs (371 pathologies); diagnoses judged by Claude Opus 4.5 at temperature 0; standalone model at temperature 0, no retrieval). Source: arxiv.org. Saturation forecast: Around August 2027. 5 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Sonnet 4.6 | 60.8 |
| 2 | GPT-5.4 | 57.6 |
| 3 | Nemotron 3 Super | 47 |
| 4 | Kimi K2.5 | 45.6 |
| 5 | GLM-5 | 27.2 |
Interactive version: theaggregate.ai/benchmark?slug=mamabench-robust-accuracy · How It Works · Data refreshed daily, snapshot 2026-09-29.