PostcondBench - Completeness: leaderboard

Comp@1 in percent: the expected share of methods whose generated postconditions pass every test and also reject every mutant of the method, for 420 documented methods from 121 Python and Java repositories (PostcondBench, 2026), one sampled postcondition set per method, averaged over code-only, comment-only and code-plus-comment inputs. Higher is better.

Metric: Comp@1 times 100 (%): expected share of methods whose generated postcondition set is test-correct and kills every mutant, one sampled postcondition set per method (k = 1, five samples drawn), averaged over the Python and Java splits (210 repository methods each) and three input settings (code only, comments only, code plus comments); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-525.5#178
2Claude Sonnet 4.5Thinking20.7#261
3Llama 4 Maverick9.4#1110
4Qwen 3 32B5.5#972
5Gemma 3 27B4.4#1663

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=postcondbench-completeness · How It Works · Data refreshed daily, snapshot 2026-11-21.