OGCaReBench (Oracle Report): leaderboard

OGCaReBench (2026) asks free-text next-step questions on rare, off-guideline clinical cases from published case reports, verified by physicians; a GPT-5.2 judge compares each answer with the step taken in the report. On this board the source case report is placed in the context; accuracy in percent, higher is better.

Metric: Accuracy (%) with the source case report of each question given in the context (oracle retrieval), on OGCaReBench (639 physician-verified free-text questions on rare, off-guideline cases from PubMed Central case reports, asking for the next clinical step after the standard options are exhausted), answers judged equivalent or mismatched to the case report's step by a GPT-5.2 judge; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Sonnet 4Thinking86.7#410
2Gemini 2.5 Pro86.5#276
3Claude Sonnet 4.584.4#261
4O4 Mini84.4#368
5Llama 3.3 70B Instruct82.5#1385
6MedGemma-27B-text-it78.7#1218
7Llama3-Med42-70B78.1#1756
8OpenBioLLM-Llama3-70B76.1#2237

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=ogcarebench-oracle-report · How It Works · Data refreshed daily, snapshot 2026-11-21.