MUSE (Text-to-CAD) - Code Execution: leaderboard

Sandbox success in percent: the share of 106 multi-part product design specifications that models turn into CadQuery code (MUSE text-to-CAD benchmark, 2026), evaluated as a funnel in which a failed stage scores zero downstream whose script runs and exports a STEP model. Higher is better.

Metric: Sandbox success (%): share of the 106 design specifications whose generated CadQuery script runs in the sandbox and exports a STEP model, one CadQuery script per design specification, executed in a sandbox; funnel evaluation, so a sample that fails a stage scores zero on every later stage; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.577.36#56
2Claude Opus 4.776.42#86
3Gemini 3.1 Pro (Preview)65.09#102
4Qwen 2.5 72B50#1189
5Claude 3.7 Sonnet46.23#455
6GPT-4o36.79#754
7GLM-5.131.13#200
8Llama 3.1 70B31.13#1509
9Qwen 3.5 122B A10B27.36#350
10MiniMax-M2.525.47#600
11MiniMax-M2.718.87#539
12MUSE Qwen3.6-Coder (checkpoint unspecified)14.15
13Qwen 3.6 35B A3B8.49#485
14GLM-4.7 Flash8.49#1349
15Llama 3.1 8B2.83#5193

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=muse-text-to-cad-code-execution · How It Works · Data refreshed daily, snapshot 2026-11-22.