O1 Preview (2024-09-12): benchmark results
September 12, 2024 O1 Preview snapshot, tracked when sources report the dated API model. Provider: OpenAI. Released 2024-09-12. Access: API.
Aggregate Rating 1700 ± 17, rank #197 of 1656 rated models, from 52 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EvalPlus | 84.6 | EvalPlus Avg. (imported) | 100 |
| KOR-Bench | 72.88 | Accuracy (%; zero-shot, 1,250 questions) | 100 |
| KOR-Bench - Cipher | 82.8 | Accuracy (%; zero-shot, 250 questions) | 100 |
| KOR-Bench - Counterfactual | 92.8 | Accuracy (%; zero-shot, 250 questions) | 100 |
| KOR-Bench - Puzzle | 36.8 | Accuracy (%; zero-shot, 250 questions) | 100 |
| MBPP+ | 80.2 | MBPP+ pass@1 (imported) | 100 |
| SAD - Anti-Imitation | 29 | Score (%, higher is better) | 100 |
| SAD - Anti-Imitation (Situating Prompt) | 29 | Score (%, higher is better) | 100 |
| SAD - Count Tokens | 19.8 | Token-count accuracy (%, higher is better) | 100 |
| SAD - Count Tokens (Situating Prompt) | 20 | Token-count accuracy (%, higher is better) | 100 |
| SAD - Influence | 84.38 | Score (%, higher is better) | 100 |
| SAD - Influence (Situating Prompt) | 93.16 | Score (%, higher is better) | 100 |
Interactive version: theaggregate.ai/model?slug=o1-preview-2024-09-12 · How It Works · Data refreshed daily, snapshot 2026-10-20.