CXR-ContraBench: leaderboard
Metric: Accuracy (%) on the CheXpert direct presence probe of CXR-ContraBench: 235 three-option questions from 82 CheXpert validation studies asking which finding is present, with the present finding, its negated form (No X) and another finding as options; base multiple-choice prompting without chain of thought or the authors' verifier, exact match; chance is 33.3; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-4o | 62.55 |
| 2 | MedGemma-27B-IT | 51.91 |
| 3 | MedGemma-4B-IT | 31.49 |
| 4 | Qwen 2.5 VL 7B Instruct | 30.21 |
Interactive version: theaggregate.ai/benchmark?slug=cxr-contrabench · How It Works · Data refreshed daily, snapshot 2026-10-07.