SADU - ER Diagrams: leaderboard

SADU (2026) tests vision-language models on counting and retrieval questions about behavioral, structural and entity-relationship software architecture diagrams. This board is exact-match accuracy in percent on questions about entity-relationship diagrams; higher is better.

Metric: Exact-match accuracy (%) on the 773 entity-relationship-diagram questions of SADU (154 software architecture diagrams (behavioral, structural and ER) with 2,431 counting and retrieval questions, temperature 0); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Flash82.54#463
2Gemini 3.1 Flash Lite (Preview)79.43#417
3Gemini 3 Flash (Preview)78.53#152
4Claude Sonnet 4.569.6#261
5GPT-5 Nano67.92#906
6Claude Haiku 4.558.86#583
7Qwen 2.5 VL 32B55.5#941
8Qwen 2.5 VL 7B51.49#1862
9Gemini 2.5 Flash Lite50.71#964
10Qwen 2.5 VL 3B38.42#2639
11GPT-4o Mini24.71#1667

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sadu-er-diagrams · How It Works · Data refreshed daily, snapshot 2026-11-21.