llm-jp-3-440m-instruct2: benchmark results
LLM-jp 3 instruction-tuned release developed by NII's LLM-jp project, using Llama architecture and a 4,096-token context window. The named instruction-tuning generation remains distinct from the pretrained checkpoint and other instruct generations. Provider: National Institute of Informatics. Access: Open.
Aggregate Rating 1264 ± 42, rank #9131 of 10916 rated models, from 62 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM - Xlsum JA Rouge1 | 26.87 | Score (%) | 59.4 |
| Open Japanese LLM - Xlsum JA RougeLsum | 21.44 | Score (%) | 45 |
| Open Japanese LLM - Xlsum JA Bert Score JA F1 | 67.7 | Score (%) | 32.8 |
| Open Japanese LLM - Wiki Coreference SET F1 | 1.41 | Score (%) | 32.1 |
| Open Japanese LLM - Xlsum JA Bleu JA | 2.21 | Score (%) | 31.4 |
| pfgen-bench - Completion Mode - Truthfulness | 0.63 | Truthfulness Score | 30.8 |
| Open Japanese LLM - Wiki NER SET F1 | 1.77 | Score (%) | 27 |
| pfgen-bench - Completion Mode - Score | 0.37 | pfgen Score (mean of three) | 26.8 |
| Open Japanese LLM - Xlsum JA Rouge2 | 7.32 | Score (%) | 26 |
| Open Japanese LLM - SUM | 7.28 | Score (%) | 25.7 |
| pfgen-bench - Completion Mode - Fluency | 0.46 | Fluency Score | 25.6 |
| pfgen-bench - QA Mode - Fluency | 0.45 | Fluency Score | 23.6 |
Interactive version: theaggregate.ai/model?slug=llm-jp-3-440m-instruct2 · How It Works · Data refreshed daily, snapshot 2026-11-08.