OLMo 2 7B Instruct November 2024: benchmark results
Provider: Allen AI. Released 2024-11-26. Access: Open.
Unified ELO 1363 ± 1, rank #1237 of 1392 rated models, from 11 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| HELM Safety HarmBench | 57.8 | LM Evaluated Safety score (%) | 23.3 |
| HELM Safety XSTest | 88.8 | LM Evaluated Safety score (%) | 10.5 |
| HELM Capabilities - GPQA | 29.6 | COT correct | 9 |
| HELM Capabilities - IFEval | 69.29 | IFEval Strict Acc | 8 |
| HELM Capabilities - MMLU-Pro | 29.2 | COT correct | 6 |
| HELM Capabilities - Omni-MATH | 11.63 | Acc | 6 |
| HELM Capabilities - WildBench | 62.84 | WB Score | 4 |
| HELM Safety | 72.9 | Mean score (self-reported) | 4 |
| HELM Safety SimpleSafetyTests | 77.5 | LM Evaluated Safety score (%) | 3.5 |
| HELM Safety Anthropic Red Team | 88 | LM Evaluated Safety score (%) | 2.3 |
| HELM Safety BBQ | 52.5 | BBQ accuracy (%) | 1.2 |
Interactive version: theaggregate.ai/model?slug=olmo-2-7b-instruct-november-2024 · How It Works · Data refreshed daily, snapshot 2026-09-05.