O1 (2024-12-17) (High): benchmark results
O1 (2024-12-17) evaluated at the high reasoning-effort setting. Provider: OpenAI. Released 2024-12-17. Access: API.
Aggregate Rating 1662 ± 17, rank #484 of 2168 rated models, from 93 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Vals AI MedQA | 96.52 | Accuracy (%) | 100 |
| PlatinumBench (MIT) | 0.74 | Category-balanced mean error rate (%) | 97 |
| Nejumi 4 - jaster (0-shot) - JCoLA (in-domain) | 82 | Exact match (%) | 96.5 |
| Nejumi 4 - jaster (0-shot) - JCommonsenseQA | 100 | Exact match (%) | 96.2 |
| Nejumi 4 - jaster (2-shot) - JaMP | 82 | Exact match (%) | 96.2 |
| Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain) | 89 | Exact match (%) | 94.8 |
| UGI - Natural Intelligence | 63.08 | NatInt Score | 93.6 |
| Nejumi 4 - BFCL - Non-Live AST | 81.48 | Accuracy (%) | 92 |
| Nejumi 4 - jaster (0-shot) - JSICK | 84 | Exact match (%) | 91.6 |
| UGI - Writing | 60.11 | Writing Score | 91.5 |
| Nejumi 4 - jaster (2-shot) - AIO | 94.79 | Character F1 (x100) | 90.9 |
| Nejumi 4 - jaster (2-shot) - JCoLA (in-domain) | 81 | Exact match (%) | 88.8 |
Interactive version: theaggregate.ai/model?slug=o1-2024-12-17-high · How It Works · Data refreshed daily, snapshot 2026-10-20.