PostcondBench - Correctness: leaderboard

Corr@1 in percent: the expected share of methods whose LLM-generated formal postconditions pass every test, for 420 documented methods from 121 Python and Java repositories (PostcondBench, 2026), one sampled postcondition set per method, averaged over code-only, comment-only and code-plus-comment inputs. Higher is better.

Metric: Corr@1 times 100 (%): expected share of methods whose generated postcondition set passes every test, one sampled postcondition set per method (k = 1, five samples drawn), averaged over the Python and Java splits (210 repository methods each) and three input settings (code only, comments only, code plus comments); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Sonnet 4.5Thinking62.9#261
2GPT-548.3#178
3Llama 4 Maverick21.4#1110
4Qwen 3 32B11.8#972
5Gemma 3 27B11#1663

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=postcondbench-correctness · How It Works · Data refreshed daily, snapshot 2026-11-21.