Llama-3.1-Swallow-8B-Instruct-v0.3: benchmark results
Provider: Institute of Science Tokyo. Released 2024-12-23. Access: Open.
Aggregate Rating 1523 ± 28, rank #2033 of 10916 rated models, from 58 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| pfgen-bench - Completion Mode - Truthfulness | 0.93 | Truthfulness Score | 97.1 |
| Open Japanese LLM - ALT E TO J Bleu JA | 14.07 | Score (%) | 91.5 |
| Open Japanese LLM - Wiki PAS SET F1 | 10.26 | Score (%) | 90.4 |
| pfgen-bench - Completion Mode - Fluency | 0.81 | Fluency Score | 89.9 |
| pfgen-bench - Completion Mode - Score | 0.7 | pfgen Score (mean of three) | 87.5 |
| Open Japanese LLM - ALT E TO J Bert Score JA F1 | 86.86 | Score (%) | 87.3 |
| pfgen-bench - QA Mode - Truthfulness | 0.93 | Truthfulness Score | 86.7 |
| Open Japanese LLM - Niilc Char F1 | 59.02 | Score (%) | 86.5 |
| Open Japanese LLM - AIO Char F1 | 73.76 | Score (%) | 86.3 |
| Open Japanese LLM - SUM | 11.21 | Score (%) | 85 |
| Open Japanese LLM - Xlsum JA Rouge2 | 11.22 | Score (%) | 85 |
| Open Japanese LLM - ALT E TO J Comet Wmt22 | 90.95 | Score (%) | 84.3 |
Interactive version: theaggregate.ai/model?slug=llama-3-1-swallow-8b-instruct-v0-3 · How It Works · Data refreshed daily, snapshot 2026-11-08.