CliniCARE-Bench - Over-Commitment Rate: leaderboard
CliniCARE-Bench (arXiv 2608.07796v1, Table 3): agent systems audit de-identified EHR records against a clinical policy corpus across 25 clinician-built scenarios (750 cases, one run each) and return Yes, No, or Indeterminate (lack of data or medically ambiguous). Share of cases whose reference verdict requires deferral on which the system still gives a definitive verdict, in percent; lower is better. Rows are harness plus model systems.
Metric: Definitive verdicts on cases whose reference requires deferral (%; lower is better). Source: arxiv.org. Saturation forecast: Around February 2027. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 21.3 | #64 |
| 2 | Claude Sonnet 5 | 30.7 | #65 |
| 3 | GPT-5.6 Sol | 31.3 | #22 |
| 4 | Claude Opus 5 | 32 | #16 |
| 5 | GPT-5.4 | 36.7 | #83 |
| 6 | Gemini 3.6 Flash | 38 | #57 |
| 7 | GPT-5.5 | 38.7 | #33 |
| 8 | GPT-5.6 Luna | 39.3 | #79 |
| 9 | DeepSeek V4 Pro | 40.7 | #112 |
| 10 | Gemini 3.5 Flash | 42 | #66 |
| 11 | GLM-5.2 | 42 | #80 |
| 12 | Qwen 3.7 Plus | 46 | #98 |
| 13 | DeepSeek V4 Flash | 52 | #155 |
| 14 | GPT-5.4 Mini | 52.7 | #228 |
| 15 | Kimi K2.7 Code | 52.7 | #95 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=clinicare-bench-over-commitment-rate · How It Works · Data refreshed daily, snapshot 2026-11-05.