OpenLayout-Bench - Standalone - Tool A Pass@5: leaderboard

Atomic physical-design task correctness, first or within5attempts; temperature0.1. ToolA/B are anonymized Innovus22/ICC22022 in arbitrary order (do not inferwhich). STA PrimeTime2022; TSMC28nm. Gemini=3.1-pro-preview. Separate common scaffolds, no barebase merging of different scaffolds. Exact Qwen3VL8B/30B variant and GPT5Thinking API IDs not supplied; preserve source labels without inventing Instruct/Thinking checkpoint bindings. Metric: Correct executable solution rate (0–1). Source: https://arxiv.org/html/2611.18517v1

Metric: Correct executable solution rate (0–1). Source: arxiv.org. Saturation forecast: Around 2030. 6 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3.1 Pro (Preview)0.222#99
2GPT-5Thinking0.222#177
3GPT-4o0.211#754
4DeepSeek V30.2#734
5Qwen 3 VL 8B0.156#919
6Qwen 3 VL 30B0.156#572

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=openlayout-bench-standalone-tool-a-pass-5 · How It Works · Data refreshed daily, snapshot 2026-11-17.