Meta-LLama-3-Cat-Smaug-LLama-70B Benchmark Results

Meta-LLama-3-Cat-Smaug-LLama-70B: benchmark results

gbueno86's SLERP merge of Smaug Llama 3 70B Instruct with the Llama 3 Cat 70B roleplay tune. Provider: Other. Released 2024-05-24. Access: Open.

Unified ELO 1596 ± 1, rank #274 of 1392 rated models, from 14 benchmark results.

Benchmark	Score	Metric	Percentile
Open Chinese LLM - GSM8K	72.93	Accuracy (%)	99.4
Open LLM Leaderboard - IFEval	80.72	Score	98.2
Open Chinese LLM - CMMLU	72.48	Accuracy (%)	96.7
Open LLM Leaderboard - BBH	51.51	Score	94.5
Open Chinese LLM Leaderboard	67.43	Average Score (%)	88.4
Open LLM Leaderboard - MMLU-Pro	45.28	Score	88.3
Open Chinese LLM - ARC Challenge	59.64	Accuracy (%)	85.6
Open Chinese LLM - C-Eval Semantic	87.44	Accuracy (%)	85.3
Open LLM Leaderboard - MATH Level 5	29.38	Score	81.4
Open LLM Leaderboard - MuSR	15	Score	81
Open LLM Leaderboard - GPQA	10.29	Score	79.2
Open Chinese LLM - HellaSwag	64.76	Accuracy (%)	77.7