Smaug-Llama-3-70B-Instruct-32K Benchmark Results — The Aggregate

Smaug-Llama-3-70B-Instruct-32K — benchmark results

Abacus.AI's Smaug fine-tune of Llama 3 70B Instruct with context extended to 32K via PoSE training. Provider: Other. Released 2024-06-11. Access: Open.

Unified ELO 1491 ± 27, rank #837 of 1776 rated models, from 14 benchmark results.

Strongest benchmark results

Benchmark	Score	Metric	Percentile
Open LLM Leaderboard - IFEval	77.61	Score	93.5
Open Chinese LLM - CMMLU	71.66	Accuracy (%)	93.2
Open Chinese LLM - GSM8K	68.08	Accuracy (%)	92.6
Open LLM Leaderboard - BBH	49.07	Score	89.7
Open LLM Leaderboard - MMLU-Pro	41.83	Score	86.5
Open Chinese LLM Leaderboard	66.05	Average Score (%)	82.5
Open LLM Leaderboard - MATH Level 5	27.49	Score	79.7
Open Chinese LLM - C-Eval Semantic	86.22	Accuracy (%)	79.1
Open Chinese LLM - ARC Challenge	57.94	Accuracy (%)	77.9
Open Chinese LLM - HellaSwag	63.79	Accuracy (%)	74.5
Open LLM Leaderboard - MuSR	12.43	Score	67.8
Open LLM Leaderboard - GPQA	6.15	Score	51.6

Interactive version: theaggregate.ai/model?slug=smaug-llama-3-70b-instruct-32k · How the rankings work · Data refreshed daily, snapshot 2026-07-22.