JMed48k (Text-Only): leaderboard

JMed48k (2026) is built from the official question booklets of 11 Japanese national healthcare licensing examinations. This board is accuracy (%) on the 9,905 text-only scored questions of the 2021-2025 exams, the unweighted mean over the 11 professions; higher is better.

Metric: Accuracy (%) on the 9,905 text-only scored JMed48k-Eval items (2021-2025 exams), unweighted macro average over the 11 professions, official MHLW gold answers, single-shot Japanese prompts at temperature 0 with reasoning disabled where the model has a switch, JSON-constrained answers; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 21 models tracked. Updated 14 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 2.5 Pro95.5#276
2GPT-595.1#178
3GPT-5 Mini90.5#364
4Claude Sonnet 489.3#410
5Gemini 2.5 FlashNon-reasoning86.6#463
6Qwen 3.5 397B A17BNon-reasoning86.5#264
7Gemma 4 31B (IT)84.4#350
8DeepSeek R183.1#482
9Grok 4.2082.4#301
10Qwen 3.5 27BNon-reasoning82#375
11Llama 4 Maverick77.7#1110
12GLM-4.6V-FlashNon-reasoning77.1#1040
13Qwen 3.5 9BNon-reasoning68.9#828
14Lingshu-32B65#1113
15MedGemma-27B-IT60.6#1350

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=jmed48k-text-only · How It Works · Data refreshed daily, snapshot 2026-11-21.