ABRA - Easy: leaderboard

Easy radiology tasks (viewer control, DICOM metadata questions, vision probes) for an agent operating an OHIF imaging viewer and an Orthanc DICOM server through 21 tools (ABRA, 2026), scored as 0.2 planning plus 0.3 execution plus 0.5 outcome on a 0-100 scale. Higher is better.

Metric: Easy-tier score (viewer control, DICOM metadata QA, vision probes), composite 0.2 Planning + 0.3 Execution + 0.5 Outcome, each normalised to 0-1, times 100; the agent operates an OHIF viewer and an Orthanc DICOM server through 21 tools, temperature 0, provider-default reasoning effort, single run; tasks that hit the turn cap without submitting score Outcome 0; higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1ABRA Qwen 3.5 (checkpoint unspecified)93
2GPT-5.4 Nano (2026-03-17)92#473
3Claude Sonnet 4.691#173
4GPT-5.4 (2026-03-05)91#92
5ABRA Gemma 4 (checkpoint unspecified)91
6Gemini 3 Flash (Preview)84#152
7Mistral Large 382#832
8Ministral 3 14B82#1870
9Gemini 3.1 Pro (Preview)80#99
10Kimi K2.579#266

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=abra-easy · How It Works · Data refreshed daily, snapshot 2026-11-21.