llm-jp-3-980m-instruct3: benchmark results
LLM-jp 3 instruction-tuned release developed by NII's LLM-jp project, using Llama architecture and a 4,096-token context window. The named instruction-tuning generation remains distinct from the pretrained checkpoint and other instruct generations. Provider: National Institute of Informatics. Access: Open.
Aggregate Rating 1235 ± 42, rank #9375 of 10916 rated models, from 58 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM - Xlsum JA Rouge1 | 25.6 | Score (%) | 50.2 |
| Open Japanese LLM - Xlsum JA RougeLsum | 21.8 | Score (%) | 47.6 |
| pfgen-bench - Completion Mode - Fluency | 0.55 | Fluency Score | 46.6 |
| pfgen-bench - Completion Mode - Truthfulness | 0.71 | Truthfulness Score | 45.8 |
| pfgen-bench - Completion Mode - Score | 0.45 | pfgen Score (mean of three) | 43.8 |
| pfgen-bench - Completion Mode - Helpfulness | 0.09 | Helpfulness Score | 41.5 |
| Open Japanese LLM - Xlsum JA Bert Score JA F1 | 68.43 | Score (%) | 39.9 |
| Open Japanese LLM - SUM | 8.35 | Score (%) | 34.9 |
| Open Japanese LLM - Xlsum JA Rouge2 | 8.35 | Score (%) | 34.7 |
| Open Japanese LLM - AIO Char F1 | 43.8 | Score (%) | 33.7 |
| Open Japanese LLM - Niilc Char F1 | 36.96 | Score (%) | 26.2 |
| Open Japanese LLM - ALT E TO J Comet Wmt22 | 85.98 | Score (%) | 25.2 |
Interactive version: theaggregate.ai/model?slug=llm-jp-3-980m-instruct3 · How It Works · Data refreshed daily, snapshot 2026-11-08.