llm-jp-3.1-1.8B: benchmark results
LLM-jp 3.1 pretrained release developed by NII's LLM-jp project, using Llama architecture and a 4,096-token context window. Provider: National Institute of Informatics. Access: Open.
Aggregate Rating 1434 ± 40, rank #4505 of 10916 rated models, from 50 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM - Wiki Dependency SET F1 | 51.53 | Score (%) | 95.9 |
| Open Japanese LLM - FA | 29.97 | Score (%) | 89.2 |
| Open Japanese LLM - Wiki PAS SET F1 | 8.51 | Score (%) | 82.2 |
| Open Japanese LLM - AIO Char F1 | 62.05 | Score (%) | 76.9 |
| Open Japanese LLM - Wiki NER SET F1 | 9.73 | Score (%) | 76.1 |
| Open Japanese LLM - Wiki Reading Char F1 | 78.69 | Score (%) | 64.9 |
| Open Japanese LLM - Jsick Exact Match | 81.14 | Score (%) | 64.8 |
| Open Japanese LLM - Wikicorpus J TO E Bleu EN | 10.88 | Score (%) | 56.5 |
| Open Japanese LLM - ALT E TO J Bert Score JA F1 | 85.75 | Score (%) | 50.2 |
| Open Japanese LLM - QA | 52.53 | Score (%) | 50 |
| Open Japanese LLM - Jsem Exact Match | 76.64 | Score (%) | 48.4 |
| Open Japanese LLM - Wikicorpus J TO E Bert Score EN F1 | 90.1 | Score (%) | 48.4 |
Interactive version: theaggregate.ai/model?slug=llm-jp-3-1-1-8b · How It Works · Data refreshed daily, snapshot 2026-11-08.