RWE-bench: leaderboard

RWE-bench (Real World Engineering Bench, Xplore, Batch 1) asks a coding agent to do physical engineering design work: 20 tasks, 9 mechanical (parts and mechanisms modelled in CAD and checked for strength, fit and motion, such as a one-piece crankshaft for a 3-cylinder diesel engine), 5 electrical (high-speed PCB routing on dense, already populated boards, such as a DDR3 memory bus with skew and impedance limits) and 6 aerospace (CFD and conjugate heat transfer from mesh to converged solution, such as the heat load on a turbine vane). Unlike SWE-bench, where a fix passes when the repository's tests pass, each design here is checked by a deterministic verifier that runs physics solvers and passes it only if every requirement in the brief is met; there is no LLM judge and no human grading. The agent gets the brief an engineer would get and a sandbox with CAD and CAE tools, and has up to 24 hours per run. Each model runs each task 5 times in its vendor's own coding agent at the highest reasoning effort it offers. This board is the average pass rate: the share of runs that pass, averaged over the 20 tasks.

Metric: Average pass rate over 20 engineering tasks, 5 runs each (%). Source: xplore.ai. Saturation forecast: Around June 2027. 15 models tracked. Updated 14 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Fable 5.2Max62#1
2GPT-6 AstraMax37#6
3Claude Opus 5.5Max29#5
4GPT-6.1 SolMax25#10
5Gemini 4 CarbonHigh20#2
6GPT-6.1 TerraMax19#21
7Gemini 4 ArgonHigh12#12
8Claude Fable 5.1Max7#15
9Kimi K3.1 ProMax1#29
10DeepSeek V4.1 FlashMax0#62
11GLM-5.3Max0#68
12Grok 4.7xHigh0#47
13Muse WonderMax0#28
14Qwen 3.8 MaxxHigh0#59
15Nemotron 3.5 Super VLMax0#349

Interactive version: theaggregate.ai/benchmark?slug=rwe-bench · How It Works · Data refreshed daily, snapshot 2026-11-16.