Meta-LLama-3-Cat-Smaug-LLama-70B Benchmark Results

Meta-LLama-3-Cat-Smaug-LLama-70B: benchmark results

gbueno86's SLERP merge of Smaug Llama 3 70B Instruct with the Llama 3 Cat 70B roleplay tune. Provider: gbueno86. Released 2024-05-24. Access: Open.

Aggregate Rating 1600 ± 46, rank #1009 of 10916 rated models, from 14 benchmark results.

Benchmark	Score	Metric	Percentile
Open Chinese LLM - GSM8K	72.93	Accuracy (%)	99.4
Open LLM Leaderboard - IFEval	80.72	Score	98.1
Open Chinese LLM - CMMLU	72.48	Accuracy (%)	96.7
Open LLM Leaderboard - BBH	66.74	Score	95.1
Open Chinese LLM Leaderboard	68.36	Official Nonzero-Task Mean (%)	90.5
Open LLM Leaderboard - MMLU-Pro	50.75	Score	88.3
Open Chinese LLM - ARC Challenge	59.64	Accuracy (%)	85.7
Open Chinese LLM - C-Eval Semantic	87.44	Accuracy (%)	85.4
Open LLM Leaderboard - MATH Level 5	29.38	Score	81.4
Open LLM Leaderboard - GPQA	32.72	Score	79.2
Open Chinese LLM - HellaSwag	64.76	Accuracy (%)	77.8
Open LLM Leaderboard - MuSR	43.68	Score	75.7