AcuityBench: leaderboard
We introduce AcuityBench, a benchmark for evaluating whether language models identify the appropriate urgency of care from user medical presentations. Existing health benchmarks emphasize medical question answering, broad health interactions, or narrow workflow-specific triage tasks, but they do not offer a unified evaluation of acuity identification across these settings. AcuityBench addresses th
Metric: QA Exact (self-reported). Source: benchmarklist.com. Saturation forecast: Around June 2027. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.7 | 85.3 | #49 |
| 2 | Gemini 2.5 Pro | 79.3 | #152 |
| 3 | GPT-5 Mini | 78 | #186 |
| 4 | GPT-5.4 | 77.2 | #79 |
| 5 | GPT-4.1 | 76.8 | #248 |
| 6 | Claude Sonnet 4.6 | 76.1 | #90 |
| 7 | DeepSeek V3.1 | 72.9 | #264 |
| 8 | Gemini 2.5 Flash | 70.8 | #245 |
| 9 | Claude Haiku 4.5 | 63 | #280 |
| 10 | Qwen 2.5 7B | 58.4 | #922 |
| 11 | Qwen 2.5 72B | 56.2 | #499 |
| 12 | Llama 3.3 70B | 53.3 | #579 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=acuitybench · How It Works · Data refreshed daily, snapshot 2026-10-20.