CompliBench - Healthcare - Guideline Accuracy: leaderboard
CompliBench (2026) evaluates LLMs as judges of guideline compliance in task-oriented dialogue: synthesized multi-turn conversations carry injected, adversarially optimized violations, and the judge must name the governing guideline and violation status of every assistant turn. This board is strict guideline accuracy in percent in the healthcare domain, the share of compliant turns for which the judge names the right guideline and no violation; higher is better.
Metric: Strict guideline accuracy (%): share of compliant assistant turns in the healthcare domain for which the judge predicts the correct governing guideline and no violation, of an LLM judge on CompliBench's synthesized multi-turn dialogues with injected, adversarially optimized guideline violations, mean of four runs at default reasoning effort; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-5 | 92.86 | #178 | |
| 2 | Gemini 3 Pro | 92.39 | #145 | |
| 3 | Qwen 3.5 Plus | 92.39 | #242 | |
| 4 | GPT-4o | 90.41 | #751 | |
| 5 | DeepSeek V3.2 | Thinking | 89.95 | #419 |
| 6 | Qwen 3 Max | 88.24 | #415 | |
| 7 | Claude Sonnet 4.6 | 87.33 | #173 | |
| 8 | GLM-5 | 86.18 | #269 | |
| 9 | Qwen 3 30B A3B | 84.55 | #1156 | |
| 10 | Kimi K2.5 | 79.35 | #266 | |
| 11 | Qwen 3 32B | 62.42 | #972 | |
| 12 | Qwen 3 4B | 54.43 | #2348 | |
| 13 | GPT-4o Mini | 51.13 | #1667 | |
| 14 | Qwen 3 8B | 49.34 | #1873 | |
| 15 | Qwen 3 14B | 40.49 | #1269 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=complibench-healthcare-guideline-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-21.