MamaBench - Base Accuracy: leaderboard
MamaBench (Helpmum Africa and University of Ibadan, 2026) pairs 217 expert-written maternal and paediatric patient narratives with minimally perturbed counterfactuals whose correct diagnosis differs, testing whether models tell clinically similar presentations apart. Base accuracy is the share of pairs whose unperturbed base case is diagnosed correctly, in percent; higher is better.
Metric: Base accuracy (%; share of pairs whose base-case diagnosis is correct; 217 expert-authored maternal and paediatric patient narratives, each paired with a minimally perturbed counterfactual whose diagnosis differs (371 pathologies); diagnoses judged by Claude Opus 4.5 at temperature 0; standalone model at temperature 0, no retrieval). Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 82 | #161 |
| 2 | GPT-5.4 | 77.4 | #123 |
| 3 | Kimi K2.5 | 76 | #256 |
| 4 | Nemotron 3 Super | 72.4 | #768 |
| 5 | GLM-5 | 54.8 | #260 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mamabench-base-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-13.