llm-jp-3-1.8B-instruct: benchmark results
LLM-jp 3 instruction-tuned release developed by NII's LLM-jp project, using Llama architecture and a 4,096-token context window. The named instruction-tuning generation remains distinct from the pretrained checkpoint and other instruct generations. Provider: National Institute of Informatics. Access: Open.
Aggregate Rating 1382 ± 51, rank #6877 of 10556 rated models, from 62 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM - AIO Char F1 | 60.03 | Score (%) | 75.5 |
| pfgen-bench - Completion Mode - Fluency | 0.62 | Fluency Score | 60.1 |
| pfgen-bench - Completion Mode - Score | 0.51 | pfgen Score (mean of three) | 58.8 |
| pfgen-bench - Completion Mode - Truthfulness | 0.77 | Truthfulness Score | 58.1 |
| pfgen-bench - Completion Mode - Helpfulness | 0.15 | Helpfulness Score | 55.9 |
| pfgen-bench - QA Mode - Fluency | 0.59 | Fluency Score | 52.1 |
| Open Japanese LLM - Wiki Coreference SET F1 | 2.88 | Score (%) | 51 |
| pfgen-bench - QA Mode - Truthfulness | 0.75 | Truthfulness Score | 50.9 |
| pfgen-bench - QA Mode - Score | 0.49 | pfgen Score (mean of three) | 48.5 |
| Open Japanese LLM - Wiki Reading Char F1 | 72.72 | Score (%) | 43 |
| pfgen-bench - QA Mode - Helpfulness | 0.14 | Helpfulness Score | 42.4 |
| Open Japanese LLM - Niilc Char F1 | 45.61 | Score (%) | 41.3 |
Interactive version: theaggregate.ai/model?slug=llm-jp-3-1-8b-instruct · How It Works · Data refreshed daily, snapshot 2026-11-17.