llm-jp-3-7.2B: benchmark results
National Institute of Informatics' LLM-jp-3 7.2B transformer model, part of Japan's LLM-jp research project, pretrained on a blend of datasets totaling 2.1 trillion tokens seen. Provider: National Institute of Informatics. Released 2024-12-01. Access: Open.
Aggregate Rating 1394 ± 45, rank #1222 of 1635 rated models, from 26 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| pfgen-bench - Completion Mode - Truthfulness | 0.85 | Truthfulness Score | 75.9 |
| pfgen-bench - Completion Mode - Fluency | 0.71 | Fluency Score | 75.2 |
| pfgen-bench - Completion Mode - Score | 0.61 | pfgen Score (mean of three) | 74.1 |
| pfgen-bench - Completion Mode - Helpfulness | 0.26 | Helpfulness Score | 73.3 |
| Swallow - Pre-trained Japanese - NIILC | 60.1 | Character F1 (%) | 67.3 |
| Swallow - Pre-trained Japanese - JEMHopQA | 48.1 | Character F1 (%) | 51 |
| Swallow - Pre-trained Japanese - WMT20 En-Ja | 24.9 | BLEU | 49 |
| Swallow - Pre-trained English - XWINO | 88.8 | Accuracy (%) | 45.9 |
| Swallow - Pre-trained Japanese - XL-Sum | 15.2 | ROUGE-2 (%) | 42.9 |
| Swallow - Pre-trained English - TriviaQA | 52.2 | Exact Match (%) | 40.8 |
| Swallow - Pre-trained Japanese - WMT20 Ja-En | 19 | BLEU | 39.8 |
| Swallow - Pre-trained English - HellaSwag | 54.4 | Accuracy (%) | 36.7 |
Interactive version: theaggregate.ai/model?slug=llm-jp-3-7-2b · How It Works · Data refreshed daily, snapshot 2026-11-05.