Humanity's Last Exam - Calibration Error: leaderboard
The column only the official CAIS/Scale HLE board publishes: how far a model's stated confidence sits from how often it is actually right. HLE accuracy is covered from twenty-odd sources; this is not that number. Lower is better, and every model on the board is above 50 — they are confidently wrong far more often than they are uncertain.
Metric: Calibration Error (%, lower is better). Source: agi.safe.ai. Saturation forecast: Around January 2027. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 50 | #98 |
| 2 | Grok 4 | 56.4 | #181 |
| 3 | Gemini 3 Pro | 57.2 | #80 |
| 4 | Claude Sonnet 4.5 | 65 | #142 |
| 5 | GPT-5 Mini | 65 | #186 |
| 6 | Gemini 2.5 Pro | 72 | #152 |
| 7 | DeepSeek R1 | 73 | #251 |
| 8 | Gemini 2.5 Flash | 80 | #245 |
| 9 | O1 | 83 | #183 |
| 10 | GPT-4o | 89 | #343 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=humanity-s-last-exam-calibration-error · How It Works · Data refreshed daily, snapshot 2026-10-20.