OGCaReBench (Oracle Report): leaderboard
OGCaReBench (2026) asks free-text next-step questions on rare, off-guideline clinical cases from published case reports, verified by physicians; a GPT-5.2 judge compares each answer with the step taken in the report. On this board the source case report is placed in the context; accuracy in percent, higher is better.
Metric: Accuracy (%) with the source case report of each question given in the context (oracle retrieval), on OGCaReBench (639 physician-verified free-text questions on rare, off-guideline cases from PubMed Central case reports, asking for the next clinical step after the standard options are exhausted), answers judged equivalent or mismatched to the case report's step by a GPT-5.2 judge; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Sonnet 4 | Thinking | 86.7 | #410 |
| 2 | Gemini 2.5 Pro | 86.5 | #276 | |
| 3 | Claude Sonnet 4.5 | 84.4 | #261 | |
| 4 | O4 Mini | 84.4 | #368 | |
| 5 | Llama 3.3 70B Instruct | 82.5 | #1385 | |
| 6 | MedGemma-27B-text-it | 78.7 | #1218 | |
| 7 | Llama3-Med42-70B | 78.1 | #1756 | |
| 8 | OpenBioLLM-Llama3-70B | 76.1 | #2237 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=ogcarebench-oracle-report · How It Works · Data refreshed daily, snapshot 2026-11-21.