Evolving Requirements - Evo+Comp StrictPass: leaderboard
61 frozen repository tasks, 207 requirement states. ClaudeCode2.1.74, 200ktoken context, high reasoning, provider-default sampling, network-disabled. Full one complete session; Evo one response/state; Evo+Comp initial plus<=2completion revisions/state; Hy4-Preview high feedbackgenerator20ktokens and<=2publicobservations/message. StrictPass allchecks; NodeCredit partialstate credit; FDAC feedback-discounted adaptivecorrectness. Pooled row withheld. Metric: Native score (%). Source: https://arxiv.org/html/2610.35488v1
Metric: Native score (%). Source: arxiv.org. Saturation forecast: Around April 2027. 12 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-6 Astra | High | 72.1 | #8 |
| 2 | GPT-5.6 Sol | High | 70.5 | #33 |
| 3 | Kimi K3 | High | 70.5 | #59 |
| 4 | DeepSeek V4 Flash | High | 70.5 | #265 |
| 5 | DeepSeek V4.1 Flash | High | 70.5 | #69 |
| 6 | Hy4 preview | High | 68.9 | #75 |
| 7 | GLM-5.3 | High | 65.6 | #73 |
| 8 | Grok 4.6 | High | 63.9 | #47 |
| 9 | GLM-5.3 Flash | High | 62.3 | #136 |
| 10 | DeepSeek V4 Pro | High | 60.7 | #198 |
| 11 | Qwen 3.8 Flash | High | 57.4 | #128 |
| 12 | Hy3 | High | 36.1 | #211 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=evolving-requirements-evo-pluscomp-strictpass · How It Works · Data refreshed daily, snapshot 2026-11-17.