llm-jp-4.1-8B: benchmark results
Provider: National Institute of Informatics. Released 2026-09-28. Access: Open.
Aggregate Rating 1614 ± 15, rank #398 of 1647 rated models, from 136 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM v2 - JCoLA Out-of-Domain | 82.87 | Accuracy (%) | 100 |
| LLM-jp4.1 Release Swallow - WMT20 JA→EN | 0.21 | BLEU (source-normalized 0–1) | 92.3 |
| Open Japanese LLM v2 - M-IFEval-Ja | 50.58 | Prompt-Level Strict Accuracy (%) | 92 |
| Nejumi 4 - BFCL - Irrelevance Detection | 93.33 | Accuracy (%) | 91.5 |
| Open Japanese LLM v2 - JCoLA In-Domain | 79.19 | Accuracy (%) | 90.9 |
| LLM-jp4.1 Release Swallow - IFBench | 0.58 | Score (0–1) | 84.6 |
| LLM-jp4.1 Release Swallow - HumanEval+ @1 | 0.91 | Pass@1 (0–1) | 76.9 |
| LLM-jp4.1 Release Swallow - MIFEval JA | 0.74 | Score (0–1) | 75 |
| LLM-jp4.1 Release Swallow - HumanEval+ @10 | 0.98 | Pass@10 (0–1) | 73.1 |
| LLM-jp4.1 Release Swallow - Polymath JA top | 0.14 | Score (0–1) | 73.1 |
| Nejumi 4 - ALT - Bias | 90.48 | Score (%) | 72.6 |
| LLM-jp4.1 Release Swallow - AIME 2026 @1 | 0.53 | Pass@1 (0–1) | 71.2 |
Interactive version: theaggregate.ai/model?slug=llm-jp-4-1-8b · How It Works · Data refreshed daily, snapshot 2026-10-17.