AA CritPt v2 - Hard Subset: leaderboard
Artificial Analysis reports CritPt v2 pass@1 on a fixed, private selection of 40 of the 70 public challenges. The subset excludes 30 questions that seven frontier models solved in all five attempts at v2 launch. It uses the full v2 agentic execution and grading protocol, then averages repeated accuracy only over those 40 tasks. Scores are percentages; the full 70-task v2 result remains separate.
Metric: Pass@1 (%). Source: artificialanalysis.ai. Saturation forecast: Around August 2027. 88 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra (High) | 71.5 | #5 (GPT-6 Astra) |
| 2 | GPT-6 Astra (Medium) | 71 | #5 (GPT-6 Astra) |
| 3 | GPT-6 Astra (Max) | 70.5 | #5 (GPT-6 Astra) |
| 4 | GPT-6 Astra (xHigh) | 70.5 | #5 (GPT-6 Astra) |
| 5 | GPT-6.1 Sol (Max) | 68.5 | #7 (GPT-6.1 Sol) |
| 6 | GPT-6.1 Sol (xHigh) | 68 | #7 (GPT-6.1 Sol) |
| 7 | GPT-6.1 Sol (High) | 67.5 | #7 (GPT-6.1 Sol) |
| 8 | GPT-6.1 Sol (Medium) | 65 | #7 (GPT-6.1 Sol) |
| 9 | GPT-6.1 Terra (Max) | 64.5 | #13 (GPT-6.1 Terra) |
| 10 | Gemini 4 Carbon (High) | 63 | #2 (Gemini 4 Carbon) |
| 11 | GPT-6.1 Terra (High) | 60.5 | #13 (GPT-6.1 Terra) |
| 12 | GPT-6 Astra (Low) | 60 | #5 (GPT-6 Astra) |
| 13 | GPT-6.1 Sol (Low) | 59.5 | #7 (GPT-6.1 Sol) |
| 14 | GPT-6.1 Terra (xHigh) | 57.5 | #13 (GPT-6.1 Terra) |
| 15 | GPT-5.6 Sol (Max) | 56 | #22 (GPT-5.6 Sol) |
No result here: #3 Claude Opus 5.5, #10 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=aa-critpt-v2-hard-subset · How It Works · Data refreshed daily, snapshot 2026-10-29.