SWE-Clar: leaderboard
A coding agent questions a simulated client about an incomplete request from a real Upwork job, then builds the software with no further contact; the score is the acceptance-test result over 100 projects, one run each, from Upwork's own runs. It scores asking the right questions up front, which full-spec coding boards do not.
Metric: Implementations passing acceptance tests, 100 tasks (%). Source: fireworks.ai. Saturation forecast: Around November 2027. 9 models tracked. Updated 16 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 67 | #6 |
| 2 | Claude Opus 5 | Max | 63 | #20 |
| 3 | Claude Fable 5.1 | Max | 60 | #15 |
| 4 | GPT-5.6 Sol | Max | 57 | #30 |
| 5 | Qwen 3.8 Max | Max | 55 | #59 |
| 6 | GLM-5.3 | Max | 54 | #68 |
| 7 | Kimi K3 | Max | 49 | #56 |
| 8 | GPT-5.6 Luna | Max | 49 | #122 |
| 9 | GLM-5.3 Flash | Max | 39 | #125 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=swe-clar · How It Works · Data refreshed daily, snapshot 2026-11-16.