Llama-3-Swallow-8B-Instruct-v0.1: benchmark results
Swallow-LLM's Llama-3-Swallow-8B-Instruct-v0.1, continually pretrained from Llama 3 8B with added Japanese data, then instruction-tuned via SFT and Chat Vector. Provider: Institute of Science Tokyo. Released 2024-07-01. Access: Open.
Aggregate Rating 1419 ± 16, rank #1153 of 1656 rated models, from 37 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-EQ-Bench | 32.3 | EQ-Bench score | 81.8 |
| Open Korean LLM - Ko-IFEval | 38.56 | Strict accuracy, prompt/instruction mean (%) | 79.9 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 42.23 | Exact match, flexible extract (%) | 77.2 |
| Open Korean LLM Leaderboard | 40.32 | Average Score (%) | 76.4 |
| Open Ko-LLM Leaderboard | 40.32 | Average (%) | 76.2 |
| Open LLM Leaderboard - MuSR | 43.57 | Score | 74.9 |
| Open LLM Leaderboard - IFEval | 55.08 | Score | 66.5 |
| pfgen-bench - Completion Mode - Truthfulness | 0.81 | Truthfulness Score | 65.7 |
| Open Korean LLM - KorNAT-Helpful | 48.13 | Normalized accuracy (%) | 64.9 |
| pfgen-bench - Completion Mode - Fluency | 0.66 | Fluency Score | 64.4 |
| pfgen-bench - Completion Mode - Score | 0.55 | pfgen Score (mean of three) | 63.9 |
| pfgen-bench - Completion Mode - Helpfulness | 0.19 | Helpfulness Score | 63.6 |
Interactive version: theaggregate.ai/model?slug=llama-3-swallow-8b-instruct-v0-1 · How It Works · Data refreshed daily, snapshot 2026-10-20.