ABRA - Easy: leaderboard
Easy radiology tasks (viewer control, DICOM metadata questions, vision probes) for an agent operating an OHIF imaging viewer and an Orthanc DICOM server through 21 tools (ABRA, 2026), scored as 0.2 planning plus 0.3 execution plus 0.5 outcome on a 0-100 scale. Higher is better.
Metric: Easy-tier score (viewer control, DICOM metadata QA, vision probes), composite 0.2 Planning + 0.3 Execution + 0.5 Outcome, each normalised to 0-1, times 100; the agent operates an OHIF viewer and an Orthanc DICOM server through 21 tools, temperature 0, provider-default reasoning effort, single run; tasks that hit the turn cap without submitting score Outcome 0; higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 10 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | ABRA Qwen 3.5 (checkpoint unspecified) | 93 | |
| 2 | GPT-5.4 Nano (2026-03-17) | 92 | #473 |
| 3 | Claude Sonnet 4.6 | 91 | #173 |
| 4 | GPT-5.4 (2026-03-05) | 91 | #92 |
| 5 | ABRA Gemma 4 (checkpoint unspecified) | 91 | |
| 6 | Gemini 3 Flash (Preview) | 84 | #152 |
| 7 | Mistral Large 3 | 82 | #832 |
| 8 | Ministral 3 14B | 82 | #1870 |
| 9 | Gemini 3.1 Pro (Preview) | 80 | #99 |
| 10 | Kimi K2.5 | 79 | #266 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=abra-easy · How It Works · Data refreshed daily, snapshot 2026-11-21.