GPT-4o Mini (2024-07-18): benchmark results
July 18, 2024 GPT-4o Mini snapshot, tracked when sources report the dated API model. Provider: OpenAI. Released 2024-07-18. Access: API.
Aggregate Rating 1566 ± 5, rank #558 of 1656 rated models, from 1427 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| AgentRewardBench (Screenshot) - AssistantBench | 80 | Precision (%) | 100 |
| HELM Image2Struct - Image2Musicsheet | 49.52 | EMS | 100 |
| HELM MedHELM - EHRSHOT | 90.67 | EM | 100 |
| KOFFVQA - Korean OCR | 100 | Score (%) | 100 |
| LLM Trustworthy - Adversarial Demo | 88.49 | Trust Score (%) | 100 |
| PyraMathBench - Numeral Recognition | 14.6 | Score (0-100) on the 714 numeral recognition items: extract | 100 |
| RECON (Long-Context Reasoning) - Cascade Propagation | 0.36 | Score (-1 to 1; tracing which downstream conclusions an inva | 100 |
| RECON (Long-Context Reasoning) - Chain Reconstruction | 0.3 | Score (-1 to 1; reconstructing multi-hop evidence chains; fu | 100 |
| RECON (Long-Context Reasoning) - Source Conflict Resolution | 0.34 | Score (-1 to 1; resolving conflicting sources; full 50K-100K | 100 |
| SWE-Refactor (Java) - Extract and Move Method | 23.24 | Successful refactorings (%) of the 142 compound Extract and | 100 |
| SWE-Refactor (Java) - Inline Method | 74.65 | Successful refactorings (%) of the 71 Inline Method instance | 100 |
| SWE-Refactor (Java) - Move Method | 22.44 | Successful refactorings (%) of the 410 Move Method instances | 100 |
Interactive version: theaggregate.ai/model?slug=gpt-4o-mini-2024-07-18 · How It Works · Data refreshed daily, snapshot 2026-10-20.