MedConclusion: leaderboard
MedConclusion (2026) tests biomedical conclusion generation: the model reads the non-conclusion sections of a PubMed structured abstract and writes the conclusion. A GPT-5.4-mini judge rates semantic similarity to the author-written conclusion from 0 to 100 on a random 30K-abstract subset; higher is better.
Metric: Semantic similarity (judge score 0-100): the model writes a formal academic conclusion (prompt A, no length or style constraint) from the non-conclusion sections of a PubMed structured abstract; a GPT-5.4-mini judge scores it 0-100 against the author-written conclusion, averaged over a random 30K-abstract subset of MedConclusion; temperature 0, at most 1,024 new tokens; higher is better. Source: arxiv.org. Saturation forecast: Around July 2028. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-5.4 | Non-reasoning | 73.22 | #138 |
| 2 | Gemini 3.1 Pro (Preview) | 71.87 | #102 | |
| 3 | Gemini 3 Flash | Non-reasoning | 71.33 | #180 |
| 4 | MiniMax-M2.1 | 71.21 | #582 | |
| 5 | Gemma 3 27B (IT) | 71.03 | #1341 | |
| 6 | GLM-4.6V | 70.86 | #721 | |
| 7 | Llama 3.1 8B Instruct | 70.53 | #4655 | |
| 8 | Qwen 3 4B 2507 Instruct | 69.8 | #2240 | |
| 9 | Kimi K2 (Thinking) | 69.79 | ||
| 10 | DeepSeek V3.2 | Non-reasoning | 69.47 | #425 |
| 11 | Gemma 2 9B (IT) | 69.31 | #2383 | |
| 12 | Qwen 2.5 VL 7B Instruct | 68.96 | #1919 | |
| 13 | DeepSeek R1 | 68.93 | #484 | |
| 14 | Qwen 2.5 7B Instruct | 66.87 | #2732 | |
| 15 | Llama 3.2 1B Instruct | 54.17 | #9033 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=medconclusion · How It Works · Data refreshed daily, snapshot 2026-11-22.