Smaug-72B-v0.1 — benchmark results
Abacus.AI's Qwen-72B-derived fine-tune using DPO-Positive, the first open-weights model to average over 80 on the Open LLM Leaderboard (February 2024). Provider: Other. Released 2024-02-02. Access: Open.
Unified ELO 1521 ± 18, rank #720 of 1776 rated models, from 14 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Chinese LLM - ARC Challenge | 67.66 | Accuracy (%) | 98.2 |
| Open Chinese LLM - TruthfulQA MC | 67.02 | Accuracy (%) | 98.2 |
| Open Chinese LLM Leaderboard | 73.16 | Average Score (%) | 98.2 |
| Open Chinese LLM - HellaSwag | 75 | Accuracy (%) | 97.9 |
| Open Chinese LLM - CMMLU | 72.32 | Accuracy (%) | 95.8 |
| Open Chinese LLM - C-Eval Semantic | 90.6 | Accuracy (%) | 95.7 |
| Open Chinese LLM - GSM8K | 70.13 | Accuracy (%) | 95.7 |
| Open Chinese LLM - WinoGrande | 69.38 | Accuracy (%) | 90.1 |
| Open LLM Leaderboard - MMLU-Pro | 40.26 | Score | 85.1 |
| Open LLM Leaderboard - BBH | 43.13 | Score | 82.3 |
| Open LLM Leaderboard - MuSR | 14.42 | Score | 78.8 |
| Open LLM Leaderboard - GPQA | 9.84 | Score | 77.5 |
Interactive version: theaggregate.ai/model?slug=smaug-72b-v0-1 · How the rankings work · Data refreshed daily, snapshot 2026-07-22.