MamaBench: leaderboard
MamaBench (Helpmum Africa and University of Ibadan, 2026) pairs 217 expert-written maternal and paediatric patient narratives with minimally perturbed counterfactuals whose correct diagnosis differs, testing whether models tell clinically similar presentations apart. The Bias Trap Rate is the share of pairs with a correct base diagnosis whose counterfactual is still answered wrongly, in percent; lower is better.
Metric: Bias Trap Rate (%; share of pairs with a correct base diagnosis whose counterfactual is still wrong; 217 expert-authored maternal and paediatric patient narratives, each paired with a minimally perturbed counterfactual whose diagnosis differs (371 pathologies); diagnoses judged by Claude Opus 4.5 at temperature 0; standalone model at temperature 0, no retrieval; lower is better). Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 25.6 | #123 |
| 2 | Claude Sonnet 4.6 | 25.8 | #161 |
| 3 | Nemotron 3 Super | 35 | #768 |
| 4 | Kimi K2.5 | 40 | #256 |
| 5 | GLM-5 | 50.4 | #260 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mamabench · How It Works · Data refreshed daily, snapshot 2026-11-13.