OGCaReBench (Oracle Report): leaderboard
Metric: Accuracy (%) with the source case report of each question given in the context (oracle retrieval), on OGCaReBench (639 physician-verified free-text questions on rare, off-guideline cases from PubMed Central case reports, asking for the next clinical step after the standard options are exhausted), answers judged equivalent or mismatched to the case report's step by a GPT-5.2 judge; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 9 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Sonnet 4 (Thinking) | 86.7 |
| 2 | Gemini 2.5 Pro | 86.5 |
| 3 | Claude Sonnet 4.5 | 84.4 |
| 4 | O4 Mini | 84.4 |
| 5 | Llama 3.3 70B Instruct | 82.5 |
| 6 | Llama3-Med42-70B | 78.1 |
Interactive version: theaggregate.ai/benchmark?slug=ogcarebench-oracle-report · How It Works · Data refreshed daily, snapshot 2026-10-07.