MedProbeBench - Holistic Quality: leaderboard

Expert-level medical guideline writing by LLMs with search tools on MedProbeBench (2026): each system writes a complete clinical guideline for 50 tumor entities from WHO classification volumes, compared with the expert guideline. Holistic quality: comprehensiveness, insight depth, standards adherence and clinical utility rated by an LLM judge, times 100; higher is better.

Metric: Holistic quality (x100): task-weighted mean of comprehensiveness, insight depth, standards adherence and clinical utility, each rated 0 to 10 by a GPT-4.1 judge against the expert guideline, for LLMs with search tools on MedProbeBench: each system writes a complete clinical guideline for 50 WHO-classification tumor entities (one run per task), compared with the expert guideline; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.273.6#185
2GPT-570#178
3Baichuan-M2-Plus68.1
4Gemini 3 Flash (Preview)67.8#152
5Baichuan-M3-Plus66.1
6Grok 465.4#319
7GPT-4.165.2#505
8Claude Sonnet 464.1#410

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=medprobebench-holistic-quality · How It Works · Data refreshed daily, snapshot 2026-11-21.