Smaug-34B-v0.1 — benchmark results
Abacus.AI's DPO-Positive fine-tune of Jon Durbin's Bagel 34B, a Yi-34B-lineage model. Provider: Other. Released 2024-01-25. Access: Open.
Unified ELO 1480 ± 19, rank #874 of 1776 rated models, from 14 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Chinese LLM - ARC Challenge | 65.87 | Accuracy (%) | 96.9 |
| Open Chinese LLM - WinoGrande | 71.59 | Accuracy (%) | 96.9 |
| Open Chinese LLM - C-Eval Semantic | 90.75 | Accuracy (%) | 96.6 |
| Open Chinese LLM - HellaSwag | 71.11 | Accuracy (%) | 95.8 |
| Open Chinese LLM Leaderboard | 69.3 | Average Score (%) | 94.1 |
| Open Chinese LLM - CMMLU | 70.02 | Accuracy (%) | 84.6 |
| Open LLM Leaderboard - MMLU-Pro | 39.37 | Score | 84.4 |
| Open Chinese LLM - TruthfulQA MC | 59.39 | Accuracy (%) | 81.6 |
| Open LLM Leaderboard - GPQA | 10.63 | Score | 80.4 |
| Open Chinese LLM - GSM8K | 56.41 | Accuracy (%) | 78.5 |
| Open LLM Leaderboard - BBH | 34.26 | Score | 67.8 |
| Open LLM Leaderboard - IFEval | 50.16 | Score | 59.2 |
Interactive version: theaggregate.ai/model?slug=smaug-34b-v0-1 · How the rankings work · Data refreshed daily, snapshot 2026-07-22.