Medical QA Uncertainty Study - MedMCQA English: leaderboard

Medical QA uncertainty study, arXiv2611.16821v1 Table2. Qwen2.5-7B-Instruct and inception42/jais-adapted-7b-chat (Jais revision f2de64b0), 4-bit NF4 with float16 compute, bitsandbytes, one NVIDIA T4, seed42. English MedMCQA filtered cohort2816questions and Arabic MedAraBench4927questions are non-parallel and stay separate. One forward pass/question, English instruction, native chat templates, no generated text. Choose maximum next-token option probability; Jais adds Arabic and Latin option-letter probabilities in both languages. No fitted-temperature/calibration/confidence/AUROC metric substituted for answer accuracy. Native accuracy fractions0–1, higher better. This board uses the MedMCQA English cohort.

Metric: Accuracy (fraction; NF4, next-token option scoring). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 2 models tracked. Updated 14 Nov 2026.

Top models

#ModelScoreOverall rank
1Qwen 2.5 7B Instruct (NF4)0.561
2jais-adapted-7B-chat (NF4)0.369

Interactive version: theaggregate.ai/benchmark?slug=medical-qa-uncertainty-study-medmcqa-english · How It Works · Data refreshed daily, snapshot 2026-11-16.