MamaBench - Robust Accuracy: leaderboard
MamaBench (Helpmum Africa and University of Ibadan, 2026) pairs 217 expert-written maternal and paediatric patient narratives with minimally perturbed counterfactuals whose correct diagnosis differs, testing whether models tell clinically similar presentations apart. Robust accuracy is the share of pairs with both the base and the counterfactual diagnosis correct, in percent; higher is better.
Metric: Robust accuracy (%; share of pairs with both the base and the counterfactual diagnosis correct; 217 expert-authored maternal and paediatric patient narratives, each paired with a minimally perturbed counterfactual whose diagnosis differs (371 pathologies); diagnoses judged by Claude Opus 4.5 at temperature 0; standalone model at temperature 0, no retrieval). Source: arxiv.org. Saturation forecast: Around May 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 60.8 | #161 |
| 2 | GPT-5.4 | 57.6 | #123 |
| 3 | Nemotron 3 Super | 47 | #768 |
| 4 | Kimi K2.5 | 45.6 | #256 |
| 5 | GLM-5 | 27.2 | #260 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mamabench-robust-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-13.