ABRA - Medium: leaderboard

Medium radiology tasks (pathology annotation with and without oracle findings, BI-RADS reporting from oracle findings) for an agent operating an OHIF imaging viewer and an Orthanc DICOM server through 21 tools (ABRA, 2026), scored as 0.2 planning plus 0.3 execution plus 0.5 outcome on a 0-100 scale. Higher is better.

Metric: Medium-tier score (pathology annotation with and without oracle findings, oracle BI-RADS reporting), composite 0.2 Planning + 0.3 Execution + 0.5 Outcome, each normalised to 0-1, times 100; the agent operates an OHIF viewer and an Orthanc DICOM server through 21 tools, temperature 0, provider-default reasoning effort, single run; tasks that hit the turn cap without submitting score Outcome 0; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash (Preview)67#152
2GPT-5.4 (2026-03-05)67#92
3Claude Sonnet 4.666#173
4Mistral Large 365#832
5Ministral 3 14B64#1870
6GPT-5.4 Nano (2026-03-17)64#473
7ABRA Gemma 4 (checkpoint unspecified)64
8ABRA Qwen 3.5 (checkpoint unspecified)63
9Kimi K2.561#266
10Gemini 3.1 Pro (Preview)59#99

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=abra-medium · How It Works · Data refreshed daily, snapshot 2026-11-21.