SADU: leaderboard

SADU (2026) tests vision-language models on counting and retrieval questions about behavioral, structural and entity-relationship software architecture diagrams. This board is exact-match accuracy in percent over all questions; higher is better.

Metric: Exact-match accuracy (%) over all questions of SADU (154 software architecture diagrams (behavioral, structural and ER) with 2,431 counting and retrieval questions, temperature 0); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash (Preview)70.18#156
2Gemini 2.5 Flash69.68#466
3Gemini 3.1 Flash Lite (Preview)66.31#419
4Claude Sonnet 4.556.36#267
5GPT-5 Nano55.45#912
6Claude Haiku 4.548.25#587
7Qwen 2.5 VL 32B45.17#944
8Gemini 2.5 Flash Lite43.03#972
9Qwen 2.5 VL 7B38.58#1878
10Qwen 2.5 VL 3B31.3#2664
11GPT-4o Mini17.77#1680

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sadu · How It Works · Data refreshed daily, snapshot 2026-11-22.