TinySwallow-1.5B: benchmark results
Sakana AI's TinySwallow-1.5B, a Japanese compact model built via TAID knowledge distillation from a Qwen2.5-32B-Instruct teacher onto a Qwen2.5-1.5B-Instruct student, then further pretrained on Japanese text. Provider: Sakana AI. Released 2024-12-26. Access: Open.
Aggregate Rating 1335 ± 38, rank #1394 of 1635 rated models, from 33 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Swallow - Pre-trained Japanese - XL-Sum | 17.3 | ROUGE-2 (%) | 56.1 |
| Swallow - Pre-trained Japanese - JCommonsenseQA | 84 | Accuracy (%) | 46.9 |
| Swallow - Pre-trained Japanese - Average | 40.2 | Average Score (%) | 40.8 |
| Swallow - Pre-trained Japanese - JMMLU | 44.6 | Accuracy (%) | 40.8 |
| Swallow - Pre-trained Japanese - NIILC | 47.4 | Character F1 (%) | 40.8 |
| Swallow - Pre-trained Japanese - MGSM | 25.6 | Exact Match (%) | 38.8 |
| Swallow - Pre-trained English - GSM8K | 37.9 | Exact Match (%) | 36.7 |
| Swallow - Pre-trained English - MMLU | 54.6 | Accuracy (%) | 34.7 |
| Swallow - Pre-trained Japanese - JHumanEval | 23.1 | Pass@1 (%) | 34.7 |
| Swallow - Pre-trained English - MATH | 16.2 | Exact Match (%) | 30.6 |
| Swallow - Pre-trained English - Average | 41.8 | Average Score (%) | 28.6 |
| pfgen-bench - Completion Mode - Helpfulness | 0.05 | Helpfulness Score | 27.8 |
Interactive version: theaggregate.ai/model?slug=tinyswallow-1-5b · How It Works · Data refreshed daily, snapshot 2026-11-05.