CareTransition-Audit: leaderboard

Discharge-summary auditing on CareTransition-Audit (2026): the model answers the 46 checklist questions of the DISCHARGED framework (Yes, No, Unclear or N/A) for each of 50 MIMIC-IV discharge summaries, and its labels are compared with clinician labels. Cohen kappa from -1 to 1; higher is better.

Metric: Cohen kappa (-1 to 1) between the model labels and the clinician labels on the 46-question DISCHARGED audit checklist applied to 50 MIMIC-IV discharge summaries (Yes, No, Unclear or N/A per question), zero-shot with six prompts per summary and an indirect chain-of-thought instruction; summary-question pairs with an N/A label or an unparseable output are left out; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Sonnet 4.50.496#261
2Gemini 3 Flash (Preview)0.483#152
3DeepSeek V3.20.423#419
4GPT-5.40.42#135
5Nova 2.0 LiteNon-reasoning0.401#1177
6Llama-3.3-nemotron-super-49B-v1.50.373#920
7Grok 4.1 Fast0.371#427
8GPT-4o0.37#751
9Llama 4 Maverick0.34#1110
10Qwen 2.5 7B Instruct0.226#2723
11Phi-40.046#1931

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=caretransition-audit · How It Works · Data refreshed daily, snapshot 2026-11-21.