SWE-Clar: leaderboard

A coding agent questions a simulated client about an incomplete request from a real Upwork job, then builds the software with no further contact; the score is the acceptance-test result over 100 projects, one run each, from Upwork's own runs. It scores asking the right questions up front, which full-spec coding boards do not.

Metric: Implementations passing acceptance tests, 100 tasks (%). Source: fireworks.ai. Saturation forecast: Around November 2027. 9 models tracked. Updated 16 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax67#6
2Claude Opus 5Max63#20
3Claude Fable 5.1Max60#15
4GPT-5.6 SolMax57#30
5Qwen 3.8 MaxMax55#59
6GLM-5.3Max54#68
7Kimi K3Max49#56
8GPT-5.6 LunaMax49#122
9GLM-5.3 FlashMax39#125

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=swe-clar · How It Works · Data refreshed daily, snapshot 2026-11-16.