Llama-3.1-Swallow-8B-v0.5: benchmark results
Provider: Meta.
Aggregate Rating 1510 ± 40, rank #2198 of 10916 rated models, from 22 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Swallow - Pre-trained English - HumanEval | 55.7 | Pass@1 (%) | 87.8 |
| Swallow - Pre-trained Japanese - NIILC | 65.7 | Character F1 (%) | 87.8 |
| Swallow - Pre-trained Japanese - XL-Sum | 21.7 | ROUGE-2 (%) | 85.7 |
| Swallow - Pre-trained Japanese - JCommonsenseQA | 95.2 | Accuracy (%) | 83.7 |
| Swallow - Pre-trained Japanese - WMT20 En-Ja | 29.4 | BLEU | 81.6 |
| Swallow - Pre-trained Japanese - JHumanEval | 49.1 | Pass@1 (%) | 77.6 |
| Swallow - Pre-trained English - TriviaQA | 66.5 | Exact Match (%) | 75.5 |
| Swallow - Pre-trained Japanese - WMT20 Ja-En | 23.2 | BLEU | 73.5 |
| Swallow - Pre-trained Japanese - Average | 54.3 | Average Score (%) | 71.4 |
| Swallow - Pre-trained Japanese - JSQuAD | 91 | Character F1 (%) | 66.3 |
| Swallow - Pre-trained English - Average | 60.5 | Average Score (%) | 65.3 |
| Swallow - Pre-trained English - XWINO | 90 | Accuracy (%) | 65.3 |
Interactive version: theaggregate.ai/model?slug=llama-3-1-swallow-8b-v0-5 · How It Works · Data refreshed daily, snapshot 2026-11-08.