Smaug-34B-v0.1-ExPO: benchmark results
Provider: Abacus.AI. Released 2024-05-29. Access: Open.
Aggregate Rating 1562 ± 46, rank #1513 of 10916 rated models, from 13 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Chinese LLM - C-Eval Semantic | 90.85 | Accuracy (%) | 97.6 |
| Open Chinese LLM - WinoGrande | 70.56 | Accuracy (%) | 94.2 |
| Open Chinese LLM - TruthfulQA MC | 62.67 | Accuracy (%) | 91.7 |
| Open Chinese LLM Leaderboard | 67.55 | Official Nonzero-Task Mean (%) | 87.9 |
| Open Chinese LLM - ARC Challenge | 60.24 | Accuracy (%) | 87.7 |
| Open Chinese LLM - CMMLU | 69.37 | Accuracy (%) | 82.5 |
| Open Portuguese LLM - BLUEX | 62.17 | Accuracy (%) | 81.8 |
| Open Portuguese LLM - OAB Exams | 51.57 | Accuracy (%) | 80.6 |
| Open Chinese LLM - GSM8K | 53.9 | Accuracy (%) | 73.8 |
| Open Portuguese LLM - FaQuAD NLI | 75.39 | Macro F1 (%) | 73.6 |
| Open Portuguese LLM - ENEM | 69.42 | Accuracy (%) | 73.2 |
| Open Portuguese LLM - ASSIN2 RTE | 91.58 | Macro F1 (%) | 68.4 |
Interactive version: theaggregate.ai/model?slug=smaug-34b-v0-1-expo · How It Works · Data refreshed daily, snapshot 2026-11-08.