llm-jp-3-150m: benchmark results
LLM-jp 3 pretrained release developed by NII's LLM-jp project, using Llama architecture and a 4,096-token context window. Provider: National Institute of Informatics. Access: Open.
Aggregate Rating 1211 ± 37, rank #9541 of 10916 rated models, from 54 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM - Mbpp Pylint Check | 1.41 | Score (%) | 27.2 |
| Open Japanese LLM - Jsick Exact Match | 60.24 | Score (%) | 19.5 |
| Open Japanese LLM - Jsem Exact Match | 67.17 | Score (%) | 17.9 |
| Open Japanese LLM - Wiki NER SET F1 | 0.88 | Score (%) | 17.2 |
| Open Japanese LLM - Xlsum JA Bleu JA | 1.5 | Score (%) | 16.3 |
| Open Japanese LLM - NLI | 52.99 | Score (%) | 15.6 |
| Open Japanese LLM - CG | 0 | Score (%) | 13 |
| Open Japanese LLM - Jnli Exact Match | 55.34 | Score (%) | 12.9 |
| pfgen-bench - Completion Mode - Truthfulness | 0.43 | Truthfulness Score | 12.8 |
| pfgen-bench - Completion Mode - Helpfulness | 0.01 | Helpfulness Score | 11.7 |
| pfgen-bench - Completion Mode - Score | 0.24 | pfgen Score (mean of three) | 10.5 |
| pfgen-bench - Completion Mode - Fluency | 0.3 | Fluency Score | 10.2 |
Interactive version: theaggregate.ai/model?slug=llm-jp-3-150m · How It Works · Data refreshed daily, snapshot 2026-11-08.