Smaug-Mixtral-v0.1 — benchmark results
Abacus.AI's Smaug-series fine-tune of Mixtral-8x7B using their DPO-Positive (DPOP) preference-training technique. Provider: Other. Released 2024-02-18. Access: Open.
Unified ELO 1420 ± 9, rank #1130 of 1776 rated models, from 14 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard - MuSR | 12.99 | Score | 71.4 |
| Open Chinese LLM - TruthfulQA MC | 56.32 | Accuracy (%) | 69.3 |
| Open LLM Leaderboard - IFEval | 55.54 | Score | 67.2 |
| Open LLM Leaderboard - BBH | 31.92 | Score | 59.2 |
| Open LLM Leaderboard - GPQA | 6.82 | Score | 57.4 |
| Open Chinese LLM - ARC Challenge | 53.16 | Accuracy (%) | 53.6 |
| Open Chinese LLM - C-Eval Semantic | 70.53 | Accuracy (%) | 52.8 |
| Open LLM Leaderboard - MMLU-Pro | 26.13 | Score | 47.6 |
| Open Chinese LLM - CMMLU | 53.02 | Accuracy (%) | 46.3 |
| Open Chinese LLM - WinoGrande | 62.59 | Accuracy (%) | 45.8 |
| Open LLM Leaderboard - MATH Level 5 | 9.52 | Score | 45.7 |
| Open Chinese LLM - HellaSwag | 59.47 | Accuracy (%) | 44.5 |
Interactive version: theaggregate.ai/model?slug=smaug-mixtral-v0-1 · How the rankings work · Data refreshed daily, snapshot 2026-07-22.