Llama-3-Swallow-8B-Instruct-v0.1 Benchmark Results

Llama-3-Swallow-8B-Instruct-v0.1 — benchmark results

Provider: Meta. Released 2024-06-26. Access: Open.

Unified ELO 1398 ± 12, rank #1238 of 1776 rated models, from 15 benchmark results.

Benchmark	Score	Metric	Percentile
Open Korean LLM Leaderboard	440.3	Average Score (%)	81.5
Open LLM Leaderboard - MuSR	13.8	Score	75.9
Open LLM Leaderboard - IFEval	55.08	Score	66.5
Open LLM Leaderboard - BBH	29.27	Score	49.2
Open Chinese LLM - TruthfulQA MC	53.34	Accuracy (%)	47.5
Open LLM Leaderboard - GPQA	5.26	Score	44
Open LLM Leaderboard - MMLU-Pro	23.2	Score	39.5
Open LLM Leaderboard - MATH Level 5	7.48	Score	39.1
Open Chinese LLM - CMMLU	50.24	Accuracy (%)	35.3
Open Chinese LLM - GSM8K	37.98	Accuracy (%)	35.3
Open Chinese LLM - C-Eval Semantic	61.95	Accuracy (%)	26.7
Open Chinese LLM Leaderboard	51.92	Average Score (%)	24.6