O1 Mini (2024-09-12) — benchmark results
September 12, 2024 O1 Mini snapshot, tracked when sources report the dated API model. Provider: OpenAI. Released 2024-09-12. Access: API.
Unified ELO 1527 ± 31, rank #697 of 1776 rated models, from 36 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EvalPlus (HumanEval+ & MBPP+) | 83.9 | Pass@1 avg (%) | 98.4 |
| ZebraLogic | 59.7 | Puzzle Accuracy (%) | 90.2 |
| HELM Safety BBQ | 96.9 | BBQ accuracy (%) | 87.2 |
| AraGen | 70.46 | 3C3H Score (%) | 83.6 |
| X-Risks Leaderboard | 20.78 | 3C3H Score (%) | 77.8 |
| HELM Safety | 95.5 | Mean score (self-reported) | 74 |
| HELM Safety HarmBench | 88.5 | LM Evaluated Safety score (%) | 72.1 |
| HELM Safety XSTest | 97 | LM Evaluated Safety score (%) | 69.2 |
| LMGame-Bench 2048 | 2757.3 | Score | 58.3 |
| LLM Arena RU | 1027 | Arena Elo | 57.1 |
| Wolfram LLM Benchmarking Project | 43.5 | Correct Functionality (%) | 56.1 |
| SuperGPQA | 45.22 | Accuracy (%) | 54.4 |
Interactive version: theaggregate.ai/model?slug=o1-mini-2024-09-12 · How the rankings work · Data refreshed daily, snapshot 2026-07-22.