RWE-bench - Pass@5: leaderboard

RWE-bench (Real World Engineering Bench, Xplore, Batch 1) asks a coding agent to do physical engineering design work: 20 tasks, 9 mechanical (parts and mechanisms modelled in CAD and checked for strength, fit and motion, such as a one-piece crankshaft for a 3-cylinder diesel engine), 5 electrical (high-speed PCB routing on dense, already populated boards, such as a DDR3 memory bus with skew and impedance limits) and 6 aerospace (CFD and conjugate heat transfer from mesh to converged solution, such as the heat load on a turbine vane). Unlike SWE-bench, where a fix passes when the repository's tests pass, each design here is checked by a deterministic verifier that runs physics solvers and passes it only if every requirement in the brief is met; there is no LLM judge and no human grading. The agent gets the brief an engineer would get and a sandbox with CAD and CAE tools, and has up to 24 hours per run. Each model runs each task 5 times in its vendor's own coding agent at the highest reasoning effort it offers. This board is pass@5: the share of the 20 tasks passed in at least 1 of the 5 runs.

Metric: Tasks passed in at least 1 of 5 runs, of 20 (%). Source: xplore.ai. Saturation forecast: Around February 2027. 15 models tracked. Updated 14 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Fable 5.2Max80#1
2Claude Opus 5.5Max70#5
3GPT-6 AstraMax60#6
4GPT-6.1 SolMax50#10
5Gemini 4 CarbonHigh45#2
6GPT-6.1 TerraMax40#21
7Gemini 4 ArgonHigh35#12
8Claude Fable 5.1Max20#15
9Kimi K3.1 ProMax5#29
10DeepSeek V4.1 FlashMax0#62
11GLM-5.3Max0#68
12Grok 4.7xHigh0#47
13Muse WonderMax0#28
14Qwen 3.8 MaxxHigh0#59
15Nemotron 3.5 Super VLMax0#349

Interactive version: theaggregate.ai/benchmark?slug=rwe-bench-pass-5 · How It Works · Data refreshed daily, snapshot 2026-11-16.