llm-jp-3-3.7B-instruct3: benchmark results

LLM-jp 3 instruction-tuned release developed by NII's LLM-jp project, using Llama architecture and a 4,096-token context window. The named instruction-tuning generation remains distinct from the pretrained checkpoint and other instruct generations. Provider: National Institute of Informatics. Access: Open.

Aggregate Rating 1415 ± 52, rank #5545 of 10556 rated models, from 58 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
Open Japanese LLM - AIO Char F167.44Score (%)80.3
Open Japanese LLM - Xlsum JA Rouge129.26Score (%)77.2
Open Japanese LLM - Xlsum JA RougeLsum24.65Score (%)74.4
pfgen-bench - QA Mode - Fluency0.7Fluency Score71.5
pfgen-bench - QA Mode - Score0.61pfgen Score (mean of three)68.5
pfgen-bench - QA Mode - Helpfulness0.3Helpfulness Score67.9
pfgen-bench - QA Mode - Truthfulness0.83Truthfulness Score66.4
Open Japanese LLM - Xlsum JA Bert Score JA F169.54Score (%)63.2
Open Japanese LLM - SUM10.08Score (%)60.1
Open Japanese LLM - Xlsum JA Rouge210.07Score (%)59.9
Open Japanese LLM - Wiki Reading Char F177.26Score (%)56.2
Open Japanese LLM - QA52.87Score (%)51.4

Interactive version: theaggregate.ai/model?slug=llm-jp-3-3-7b-instruct3 · How It Works · Data refreshed daily, snapshot 2026-11-17.