RWE-bench - Pass@5: leaderboard
RWE-bench (Real World Engineering Bench, Xplore, Batch 1) asks a coding agent to do physical engineering design work: 20 tasks, 9 mechanical (parts and mechanisms modelled in CAD and checked for strength, fit and motion, such as a one-piece crankshaft for a 3-cylinder diesel engine), 5 electrical (high-speed PCB routing on dense, already populated boards, such as a DDR3 memory bus with skew and impedance limits) and 6 aerospace (CFD and conjugate heat transfer from mesh to converged solution, such as the heat load on a turbine vane). Unlike SWE-bench, where a fix passes when the repository's tests pass, each design here is checked by a deterministic verifier that runs physics solvers and passes it only if every requirement in the brief is met; there is no LLM judge and no human grading. The agent gets the brief an engineer would get and a sandbox with CAD and CAE tools, and has up to 24 hours per run. Each model runs each task 5 times in its vendor's own coding agent at the highest reasoning effort it offers. This board is pass@5: the share of the 20 tasks passed in at least 1 of the 5 runs.
Metric: Tasks passed in at least 1 of 5 runs, of 20 (%). Source: xplore.ai. Saturation forecast: Around February 2027. 15 models tracked. Updated 14 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 80 | #1 |
| 2 | Claude Opus 5.5 | Max | 70 | #5 |
| 3 | GPT-6 Astra | Max | 60 | #6 |
| 4 | GPT-6.1 Sol | Max | 50 | #10 |
| 5 | Gemini 4 Carbon | High | 45 | #2 |
| 6 | GPT-6.1 Terra | Max | 40 | #21 |
| 7 | Gemini 4 Argon | High | 35 | #12 |
| 8 | Claude Fable 5.1 | Max | 20 | #15 |
| 9 | Kimi K3.1 Pro | Max | 5 | #29 |
| 10 | DeepSeek V4.1 Flash | Max | 0 | #62 |
| 11 | GLM-5.3 | Max | 0 | #68 |
| 12 | Grok 4.7 | xHigh | 0 | #47 |
| 13 | Muse Wonder | Max | 0 | #28 |
| 14 | Qwen 3.8 Max | xHigh | 0 | #59 |
| 15 | Nemotron 3.5 Super VL | Max | 0 | #349 |
Interactive version: theaggregate.ai/benchmark?slug=rwe-bench-pass-5 · How It Works · Data refreshed daily, snapshot 2026-11-16.