MINT (Medical) - MedMCQA (Question First, Initial Answer): leaderboard
MINT (2026) turns medical multiple-choice diagnosis cases into conversations that reveal one clinically labelled evidence shard per turn. In the Ask-Question-First setting the question comes first with an instruction to wait for enough evidence; this board scores the first answer the model commits to on the MedMCQA cases, over the cases it answered. Accuracy in percent; higher is better.
Metric: Initial-answer accuracy (%) on the 174 MedMCQA-derived diagnosis cases: the multiple-choice question comes first with an instruction to wait, evidence shards follow one per turn, and the first answer the model commits to is scored over the cases it answered (abstentions are left out of the denominator; Ask-Question-First); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 89.6 | #173 |
| 2 | O4 Mini | 86.1 | #368 |
| 3 | GPT-5 Mini | 84.4 | #364 |
| 4 | MINT MedGemma-27B (checkpoint unspecified) | 78.5 | |
| 5 | GPT-OSS-20B | 75.7 | #1183 |
| 6 | Qwen 3 4B | 58.7 | #2348 |
| 7 | Med42-v2-70B | 50.6 | |
| 8 | Med42-v2-8B | 50 | #2558 |
| 9 | MINT Qwen2.5-3B (checkpoint unspecified) | 47.5 | |
| 10 | MedGemma 1.5 4B | 37.8 | #2716 |
| 11 | MINT OLMo-2-7B (checkpoint unspecified) | 24.1 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=mint-medical-medmcqa-question-first-initial-answer · How It Works · Data refreshed daily, snapshot 2026-11-21.