sarashina2.2-1B: benchmark results
SB Intuitions' 1B Japanese-English-code base model, pretrained on 10 trillion tokens then refined with synthetic math and coding data, not yet instruction-tuned. Provider: SB Intuitions. Released 2025-03-03. Access: Open.
Aggregate Rating 1388 ± 43, rank #1240 of 1635 rated models, from 26 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| pfgen-bench - Completion Mode - Fluency | 0.69 | Fluency Score | 72.2 |
| pfgen-bench - Completion Mode - Truthfulness | 0.83 | Truthfulness Score | 70.3 |
| pfgen-bench - Completion Mode - Score | 0.58 | pfgen Score (mean of three) | 69.6 |
| pfgen-bench - Completion Mode - Helpfulness | 0.22 | Helpfulness Score | 67.6 |
| Swallow - Pre-trained Japanese - MGSM | 38.8 | Exact Match (%) | 49 |
| Swallow - Pre-trained Japanese - NIILC | 52.3 | Character F1 (%) | 46.9 |
| Swallow - Pre-trained English - HumanEval | 34.2 | Pass@1 (%) | 44.9 |
| Swallow - Pre-trained English - GSM8K | 40.3 | Exact Match (%) | 38.8 |
| Swallow - Pre-trained English - MATH | 20.6 | Exact Match (%) | 37.8 |
| Swallow - Pre-trained Japanese - Average | 39.2 | Average Score (%) | 34.7 |
| Swallow - Pre-trained Japanese - JEMHopQA | 46.2 | Character F1 (%) | 34.7 |
| Swallow - Pre-trained Japanese - WMT20 En-Ja | 21.9 | BLEU | 34.7 |
Interactive version: theaggregate.ai/model?slug=sarashina2-2-1b · How It Works · Data refreshed daily, snapshot 2026-11-05.