CodeSpecWeave-Bench: leaderboard
CodeSpecWeave-Bench (arXiv 2610.40256): 100 repository coding tasks whose requests embed 1,044 interdependent specifications checked by executable tests. Unlike SWE-bench-style issues, the specifications constrain one another, so a locally correct edit can still fail; pass rate over three samples per task, agents run in Codex or Claude Code.
Metric: Pass rate (%). Source: arxiv.org. Saturation forecast: Around September 2027. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra | 74 | #6 |
| 2 | Claude Opus 5 | 71.1 | #16 |
| 3 | GPT-5.6 Sol | 69 | #22 |
| 4 | GPT-5.5 | 67 | #33 |
| 5 | GPT-5.4 | 66 | #83 |
| 6 | Qwen 3.8 Max | 65 | #34 |
| 7 | Kimi K3 | 55 | #35 |
No result here: #3 Claude Opus 5.5, #7 GPT-6.1 Sol, #9 Gemini 4 Argon.
Interactive version: theaggregate.ai/benchmark?slug=codespecweave-bench · How It Works · Data refreshed daily, snapshot 2026-11-05.