llm-jp-3-980m-instruct2: benchmark results

LLM-jp 3 instruction-tuned release developed by NII's LLM-jp project, using Llama architecture and a 4,096-token context window. The named instruction-tuning generation remains distinct from the pretrained checkpoint and other instruct generations. Provider: National Institute of Informatics. Access: Open.

Aggregate Rating 1287 ± 52, rank #8809 of 10556 rated models, from 62 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
Open Japanese LLM - Xlsum JA Rouge127.39Score (%)63.9
Open Japanese LLM - Xlsum JA RougeLsum22.97Score (%)61.2
pfgen-bench - Completion Mode - Truthfulness0.72Truthfulness Score49
pfgen-bench - Completion Mode - Fluency0.56Fluency Score48.2
pfgen-bench - Completion Mode - Score0.46pfgen Score (mean of three)47
Open Japanese LLM - Xlsum JA Bert Score JA F168.61Score (%)43.9
pfgen-bench - Completion Mode - Helpfulness0.09Helpfulness Score43.1
Open Japanese LLM - Xlsum JA Rouge29Score (%)40.9
Open Japanese LLM - SUM9Score (%)40.7
pfgen-bench - QA Mode - Fluency0.53Fluency Score38.2
pfgen-bench - QA Mode - Score0.44pfgen Score (mean of three)36.4
pfgen-bench - QA Mode - Helpfulness0.1Helpfulness Score35.2

Interactive version: theaggregate.ai/model?slug=llm-jp-3-980m-instruct2 · How It Works · Data refreshed daily, snapshot 2026-11-17.