llm-jp-4.1-8B (Thinking): benchmark results
Provider: Other. Released 2026-09-28. Access: Open.
Aggregate Rating 1555 ± 21, rank #850 of 2159 rated models, from 38 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM v2 - JCoLA Out-of-Domain | 82.87 | Accuracy (%) | 100 |
| Open Japanese LLM v2 - M-IFEval-Ja | 50.58 | Prompt-Level Strict Accuracy (%) | 92 |
| Open Japanese LLM v2 - JCoLA In-Domain | 79.19 | Accuracy (%) | 90.9 |
| Nejumi 4 - ALT - Bias | 90.48 | Score (%) | 72 |
| Open Japanese LLM v2 - StructEval | 79.03 | StructEval Score (%) | 63.6 |
| Open Japanese LLM v2 - JCulture-MCQ | 48.7 | Set F1 (%) | 55.6 |
| Open Japanese LLM v2 - JBLiMP | 82.48 | Accuracy (%) | 54.5 |
| Open Japanese LLM v2 - JFinQA | 60.9 | Accuracy (%) | 45 |
| Nejumi 4 - jaster (0-shot) | 79.42 | Score (%) | 39.9 |
| Open Japanese LLM v2 - JamC-QA | 48.25 | Exact Match (%) | 39.1 |
| Nejumi 4 - GLP - Mathematical Reasoning | 92.5 | Score (%) | 35.8 |
| Nejumi 4 - ALT - Robustness | 83.5 | Score (%) | 33.2 |
Interactive version: theaggregate.ai/model?slug=llm-jp-4-1-8b-thinking · How It Works · Data refreshed daily, snapshot 2026-10-18.