ABRA - Medium: leaderboard
Medium radiology tasks (pathology annotation with and without oracle findings, BI-RADS reporting from oracle findings) for an agent operating an OHIF imaging viewer and an Orthanc DICOM server through 21 tools (ABRA, 2026), scored as 0.2 planning plus 0.3 execution plus 0.5 outcome on a 0-100 scale. Higher is better.
Metric: Medium-tier score (pathology annotation with and without oracle findings, oracle BI-RADS reporting), composite 0.2 Planning + 0.3 Execution + 0.5 Outcome, each normalised to 0-1, times 100; the agent operates an OHIF viewer and an Orthanc DICOM server through 21 tools, temperature 0, provider-default reasoning effort, single run; tasks that hit the turn cap without submitting score Outcome 0; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 10 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 67 | #152 |
| 2 | GPT-5.4 (2026-03-05) | 67 | #92 |
| 3 | Claude Sonnet 4.6 | 66 | #173 |
| 4 | Mistral Large 3 | 65 | #832 |
| 5 | Ministral 3 14B | 64 | #1870 |
| 6 | GPT-5.4 Nano (2026-03-17) | 64 | #473 |
| 7 | ABRA Gemma 4 (checkpoint unspecified) | 64 | |
| 8 | ABRA Qwen 3.5 (checkpoint unspecified) | 63 | |
| 9 | Kimi K2.5 | 61 | #266 |
| 10 | Gemini 3.1 Pro (Preview) | 59 | #99 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=abra-medium · How It Works · Data refreshed daily, snapshot 2026-11-21.