O1 (2024-12-17) (High): benchmark results
O1 (2024-12-17) evaluated at the high reasoning-effort setting. Provider: OpenAI. Released 2024-12-17. Access: API.
Unified ELO 1593 ± 1, rank #464 of 1761 rated models, from 27 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Vals AI MedQA | 96.52 | Accuracy (%) | 100 |
| UGI - Natural Intelligence | 63.08 | NatInt Score | 95.3 |
| UGI - Writing | 60.11 | Writing Score | 92.9 |
| Arena-Hard v2 (GPT-4.1 Judge) | 58.7 | Win Rate (%) | 85.2 |
| MATH Level 5 | 94.71 | Accuracy (%) | 84.3 |
| Arena-Hard v2 | 61 | Win Rate (%) | 77.8 |
| Vals AI TaxEval v2 | 74.28 | Accuracy (%) | 73.6 |
| Aider polyglot coding leaderboard | 61.7 | Pass rate (%) | 71.2 |
| Arena-Hard Creative Writing | 59.9 | Win Rate (%) | 66.7 |
| Vals AI MATH 500 | 90.4 | Accuracy (%) | 63.4 |
| OTIS Mock AIME 2024-25 | 73.33 | Accuracy (%) | 58.3 |
| UGI Leaderboard | 37.24 | UGI Score | 58.1 |
Interactive version: theaggregate.ai/model?slug=o1-2024-12-17-high · How It Works · Data refreshed daily, snapshot 2026-09-05.