MUSE (Text-to-CAD) - Geometric Validity: leaderboard

Geometric validity in percent: the share of 106 multi-part product design specifications that models turn into CadQuery code (MUSE text-to-CAD benchmark, 2026), evaluated as a funnel in which a failed stage scores zero downstream whose exported model is watertight, manifold, free of self-intersections and free of overlapping parts. Higher is better.

Metric: Geometric validity (%): share of the 106 design specifications whose exported STEP model passes all four binary geometry checks (watertight, manifold, self-intersection free, overlap free), one CadQuery script per design specification, executed in a sandbox; funnel evaluation, so a sample that fails a stage scores zero on every later stage; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.568.87#54
2Gemini 3.1 Pro (Preview)58.49#99
3Claude Opus 4.758.49#85
4GLM-5.127.36#198
5Claude 3.7 Sonnet23.58#470
6Llama 3.1 70B23.58#1499
7Qwen 2.5 72B20.75#1188
8GPT-4o13.21#751
9Qwen 3.5 122B A10B13.21#346
10MiniMax-M2.710.38#539
11MiniMax-M2.510.38#600
12MUSE Qwen3.6-Coder (checkpoint unspecified)10.38
13Qwen 3.6 35B A3B5.66#481
14GLM-4.7 Flash5.66#1349
15Llama 3.1 8B1.89#5196

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=muse-text-to-cad-geometric-validity · How It Works · Data refreshed daily, snapshot 2026-11-21.