llm-jp-3-1.8B-instruct: benchmark results

LLM-jp 3 instruction-tuned release developed by NII's LLM-jp project, using Llama architecture and a 4,096-token context window. The named instruction-tuning generation remains distinct from the pretrained checkpoint and other instruct generations. Provider: National Institute of Informatics. Access: Open.

Aggregate Rating 1382 ± 51, rank #6877 of 10556 rated models, from 62 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
Open Japanese LLM - AIO Char F160.03Score (%)75.5
pfgen-bench - Completion Mode - Fluency0.62Fluency Score60.1
pfgen-bench - Completion Mode - Score0.51pfgen Score (mean of three)58.8
pfgen-bench - Completion Mode - Truthfulness0.77Truthfulness Score58.1
pfgen-bench - Completion Mode - Helpfulness0.15Helpfulness Score55.9
pfgen-bench - QA Mode - Fluency0.59Fluency Score52.1
Open Japanese LLM - Wiki Coreference SET F12.88Score (%)51
pfgen-bench - QA Mode - Truthfulness0.75Truthfulness Score50.9
pfgen-bench - QA Mode - Score0.49pfgen Score (mean of three)48.5
Open Japanese LLM - Wiki Reading Char F172.72Score (%)43
pfgen-bench - QA Mode - Helpfulness0.14Helpfulness Score42.4
Open Japanese LLM - Niilc Char F145.61Score (%)41.3

Interactive version: theaggregate.ai/model?slug=llm-jp-3-1-8b-instruct · How It Works · Data refreshed daily, snapshot 2026-11-17.