MedConclusion: leaderboard

MedConclusion (2026) tests biomedical conclusion generation: the model reads the non-conclusion sections of a PubMed structured abstract and writes the conclusion. A GPT-5.4-mini judge rates semantic similarity to the author-written conclusion from 0 to 100 on a random 30K-abstract subset; higher is better.

Metric: Semantic similarity (judge score 0-100): the model writes a formal academic conclusion (prompt A, no length or style constraint) from the non-conclusion sections of a PubMed structured abstract; a GPT-5.4-mini judge scores it 0-100 against the author-written conclusion, averaged over a random 30K-abstract subset of MedConclusion; temperature 0, at most 1,024 new tokens; higher is better. Source: arxiv.org. Saturation forecast: Around July 2028. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5.4Non-reasoning73.22#138
2Gemini 3.1 Pro (Preview)71.87#102
3Gemini 3 FlashNon-reasoning71.33#180
4MiniMax-M2.171.21#582
5Gemma 3 27B (IT)71.03#1341
6GLM-4.6V70.86#721
7Llama 3.1 8B Instruct70.53#4655
8Qwen 3 4B 2507 Instruct69.8#2240
9Kimi K2 (Thinking)69.79
10DeepSeek V3.2Non-reasoning69.47#425
11Gemma 2 9B (IT)69.31#2383
12Qwen 2.5 VL 7B Instruct68.96#1919
13DeepSeek R168.93#484
14Qwen 2.5 7B Instruct66.87#2732
15Llama 3.2 1B Instruct54.17#9033

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=medconclusion · How It Works · Data refreshed daily, snapshot 2026-11-22.