O1 Mini (2024-09-12): benchmark results
September 12, 2024 O1 Mini snapshot, tracked when sources report the dated API model. Provider: OpenAI. Released 2024-09-12. Access: API.
Unified ELO 1511 ± 1, rank #645 of 1392 rated models, from 39 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EvalPlus (HumanEval+ & MBPP+) | 83.9 | Pass@1 avg (%) | 98.4 |
| HumanEval+ | 89 | HumanEval+ pass@1 (self-reported) | 97.8 |
| EvalPlus | 83.9 | EvalPlus Avg. (self-reported) | 95.8 |
| MBPP+ | 78.8 | MBPP+ pass@1 (self-reported) | 95.8 |
| ZebraLogic | 59.7 | Puzzle Accuracy (%) | 90.2 |
| HELM Safety BBQ | 96.9 | BBQ accuracy (%) | 87.2 |
| AraGen | 70.46 | 3C3H Score (%) | 83.6 |
| X-Risks Leaderboard | 20.78 | 3C3H Score (%) | 77.8 |
| HELM Safety | 95.53 | Mean score (self-reported) | 75 |
| TextClass Benchmark | 1626.65 | Meta-Elo (self-reported) | 73.8 |
| HELM Safety HarmBench | 88.5 | LM Evaluated Safety score (%) | 72.1 |
| HELM Safety XSTest | 97 | LM Evaluated Safety score (%) | 69.2 |
Interactive version: theaggregate.ai/model?slug=o1-mini-2024-09-12 · How It Works · Data refreshed daily, snapshot 2026-09-05.