Llama-3.1-Swallow-8B-Instruct-v0.2 Benchmark Results

Llama-3.1-Swallow-8B-Instruct-v0.2: benchmark results

Provider: Institute of Science Tokyo. Released 2024-11-11. Access: Open.

Aggregate Rating 1535 ± 31, rank #1888 of 10916 rated models, from 58 benchmark results.

Benchmark	Score	Metric	Percentile
pfgen-bench - Completion Mode - Truthfulness	0.93	Truthfulness Score	96.6
pfgen-bench - Completion Mode - Fluency	0.84	Fluency Score	94.2
Open Japanese LLM - ALT E TO J Comet Wmt22	91.15	Score (%)	92.8
Open Japanese LLM - ALT E TO J Bert Score JA F1	87.09	Score (%)	91.5
pfgen-bench - Completion Mode - Score	0.71	pfgen Score (mean of three)	90.7
Open Japanese LLM - Wiki PAS SET F1	10.22	Score (%)	90.1
Open Japanese LLM - ALT E TO J Bleu JA	13.83	Score (%)	89.3
Open Japanese LLM - AIO Char F1	76.29	Score (%)	88.5
Open Japanese LLM - FA	29.51	Score (%)	88
Open Japanese LLM - Niilc Char F1	60.66	Score (%)	88
Open Japanese LLM - QA	64.28	Score (%)	86.8
Open Japanese LLM - Wikicorpus J TO E Bert Score EN F1	90.92	Score (%)	86.7