CaseReportBench (Zero-Shot): leaderboard
Dense clinical information extraction from 138 open-access case reports on inborn errors of metabolism, annotated by clinical specialists in 14 categories such as history, laboratory and imaging, and neurology (CaseReportBench, Zhang et al., CHIL 2025). The model extracts each category with its own zero-shot prompt applied to the report sections that match it (filtered category-specific prompting), and the output is scored against the expert annotation with Token Set Ratio, 0-100, higher is better. Open models ran 4-bit quantized through Ollama.
Metric: Token Set Ratio (%; expert-annotated case reports, Filtered Category-Specific Prompting with zero-shot category prompts, open models 4-bit quantized). Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4o | 31.11 | #741 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=casereportbench-zero-shot · How It Works · Data refreshed daily, snapshot 2026-11-12.