ClassEval-Pro: leaderboard
ClassEval-Pro (2026) asks LLMs to implement complete Python classes from a skeleton, over 300 tasks in 11 domains composed from post-2025 GitHub code, generating each class in one pass. Class-level pass@1 (%), all of the task's tests must pass; higher is better.
Metric: Class-level full-success pass@1 (%): the generated class passes all of the task's method-level and class-level tests, on ClassEval-Pro's 300 class-level Python tasks (11 domains, single- and cross-domain class compositions mined from GitHub code created after January 2025), holistic generation (the whole class from its skeleton in one call), unbiased pass@1 from five samples at temperature 0.2, run against each task's test suite (over 90% line coverage) with a 60-second timeout; higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 5 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 Coder 480B A35B Instruct | 45.6 | #655 |
| 2 | Kimi K2 | 45.1 | #448 |
| 3 | Gemini 2.5 Pro | 44.7 | #280 |
| 4 | Qwen 3 30B A3B | 40.5 | #1165 |
| 5 | GPT-5.1 | 27.9 | #248 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=classeval-pro · How It Works · Data refreshed daily, snapshot 2026-11-22.