Llama-3.1-Swallow-8B-Instruct-v0.1: benchmark results
Provider: Institute of Science Tokyo. Released 2024-10-08. Access: Open.
Aggregate Rating 1528 ± 29, rank #1982 of 10916 rated models, from 58 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| pfgen-bench - Completion Mode - Truthfulness | 0.92 | Truthfulness Score | 93.6 |
| pfgen-bench - Completion Mode - Fluency | 0.82 | Fluency Score | 90.4 |
| Open Japanese LLM - ALT E TO J Bert Score JA F1 | 86.9 | Score (%) | 88.5 |
| Open Japanese LLM - Wikicorpus J TO E Bert Score EN F1 | 90.94 | Score (%) | 87.6 |
| Open Japanese LLM - ALT E TO J Bleu JA | 13.71 | Score (%) | 87.1 |
| Open Japanese LLM - AIO Char F1 | 73.86 | Score (%) | 86.5 |
| Open Japanese LLM - ALT E TO J Comet Wmt22 | 90.97 | Score (%) | 86.3 |
| pfgen-bench - Completion Mode - Score | 0.69 | pfgen Score (mean of three) | 86.3 |
| Open Japanese LLM - QA | 62.65 | Score (%) | 86.1 |
| Open Japanese LLM - RC | 91.77 | Score (%) | 85.8 |
| Open Japanese LLM - Wiki Reading Char F1 | 85.56 | Score (%) | 85.8 |
| Open Japanese LLM - Niilc Char F1 | 57.88 | Score (%) | 85.3 |
Interactive version: theaggregate.ai/model?slug=llama-3-1-swallow-8b-instruct-v0-1 · How It Works · Data refreshed daily, snapshot 2026-11-08.