CodeSpecBench-Func: leaderboard

CodeSpecBench (2026) asks models to write executable precondition and postcondition functions that specify the intended behaviour of a program; the function-level set uses 2,494 LeetCode problems with online-judge-validated valid and invalid tests. Pass rate, the share of specifications that accept every valid test and reject every invalid one, in percent; higher is better.

Metric: Pass rate (%), the share of generated executable pre/postcondition specifications that are both correct (accept every valid test) and complete (reject every invalid test), on CodeSpecBench-Func (2,494 LeetCode problems; 50 valid and 50 invalid inputs plus valid and model-generated buggy outputs per problem, validated by the online judge); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5 Mini47#364
2Gemini 2.5 Pro46.2#276
3GPT-OSS-120B42.5#704
4Claude Sonnet 4.5Thinking40.5#261
5GPT-540.2#178
6GPT-OSS-20B37.7#1183
7Gemini 2.5 Flash36.5#463
8QwQ-32B28.7#907
9Qwen 3 14B26.4#1269
10Qwen 3 32B25.8#972
11DeepSeek V3.2Non-reasoning24.3#419
12Qwen 3 8B16.8#1873
13Qwen 3 4B15.5#2348
14Qwen 3 1.7B1.2#5860
15Qwen 3 0.6B0.2#8350

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=codespecbench-func · How It Works · Data refreshed daily, snapshot 2026-11-21.