romulus-mistral-nemo-12B-simpo: benchmark results
Axolotl AI's SimPO preference-optimization fine-tune of Mistral Nemo 12B, trained on Princeton's gemma2-ultrafeedback-armorm pairs. Provider: Other. Released 2024-07-24. Access: Open.
Unified ELO 1566 ± 1, rank #427 of 1553 rated models, from 7 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM Leaderboard | 52.29 | Average Score (%) | 99 |
| Open LLM Leaderboard - IFEval | 60.79 | Score | 72.8 |
| Open LLM Leaderboard - MuSR | 12.98 | Score | 71.3 |
| Open LLM Leaderboard - BBH | 34.64 | Score | 69 |
| Open LLM Leaderboard - MATH Level 5 | 11.4 | Score | 51.8 |
| Open LLM Leaderboard - MMLU-Pro | 27.43 | Score | 50.4 |
| Open LLM Leaderboard - GPQA | 3.8 | Score | 33.5 |
Interactive version: theaggregate.ai/model?slug=romulus-mistral-nemo-12b-simpo · How It Works · Data refreshed daily, snapshot 2026-09-08.