CodeGolf Bench - Python Best Percentile: leaderboard

CodeGolf Bench Python Best Percentile places the shortest correct Python solution an LLM produces for each code.golf problem within the distribution of human solutions, from 0 (failing or worse than every human) to 100 (shorter than every human), and averages over problems (CodeGolf Bench). Higher is better.

Metric: Best percentile (0-100): for each problem, the best of the sampled Python solutions placed in the distribution of human code.golf solutions by character count (0 = worse than every human or failing, 100 = shorter than every human), averaged over problems; 115 code.golf problems (holes), solutions checked against the platform's hidden tests through its API; golfing prompt, temperature 0.2, top-p 0.95, up to 32,768 output tokens, 10 samples per problem; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 9 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Pro (Preview 03-25)68.04#241
2DeepSeek R167.09#482
3Llama 4 Maverick55.31#1110
4DeepSeek V3 (0324)51.6#715
5Gemini 2.5 Flash (Preview 04-17)46.42#453
6DeepSeek R1 Distill Llama 70B42.33#1669
7Qwen 3 235B A22B41.64#628
8Llama 4 Scout36.69#1818
9Gemma 3 27B (IT)25.23#1324

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=codegolf-bench-python-best-percentile · How It Works · Data refreshed daily, snapshot 2026-11-21.