OGCaReBench (RAG Top-3): leaderboard

OGCaReBench (2026) asks free-text next-step questions on rare, off-guideline clinical cases from published case reports, verified by physicians; a GPT-5.2 judge compares each answer with the step taken in the report. On this board the top 3 case reports retrieved by BGE-large from a 53,617-report corpus are placed in the context; accuracy in percent, higher is better.

Metric: Accuracy (%) with the top 3 case reports retrieved by BGE-large-en-v1.5 from the 53,617-report corpus placed in the context, on OGCaReBench (639 physician-verified free-text questions on rare, off-guideline cases from PubMed Central case reports, asking for the next clinical step after the standard options are exhausted), answers judged equivalent or mismatched to the case report's step by a GPT-5.2 judge; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1O4 Mini77.8#368
2Claude Sonnet 4Thinking76.1#410
3Gemini 2.5 Pro74.3#276
4Claude Sonnet 4.573.7#261
5Llama 3.3 70B Instruct69.5#1385
6MedGemma-27B-text-it64#1218
7Llama3-Med42-70B59.2#1756
8OpenBioLLM-Llama3-70B47.6#2237

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=ogcarebench-rag-top-3 · How It Works · Data refreshed daily, snapshot 2026-11-21.