llm-jp-4.1-33B (Thinking): benchmark results
Provider: Other. Released 2026-09-28. Access: Open.
Aggregate Rating 1597 ± 20, rank #718 of 2159 rated models, from 38 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM v2 - JBLiMP | 85.8 | Accuracy (%) | 100 |
| Open Japanese LLM v2 - M-IFEval-Ja | 53.49 | Prompt-Level Strict Accuracy (%) | 96 |
| Open Japanese LLM v2 - JCulture-MCQ | 55.15 | Set F1 (%) | 88.9 |
| Open Japanese LLM v2 - JamC-QA | 55.61 | Exact Match (%) | 87 |
| Open Japanese LLM v2 - JCoLA In-Domain | 78.73 | Accuracy (%) | 81.8 |
| Open Japanese LLM v2 - StructEval | 80.27 | StructEval Score (%) | 72.7 |
| Nejumi 4 - ALT - Ethics & Morality | 95 | Score (%) | 66.8 |
| Open Japanese LLM v2 - JCoLA Out-of-Domain | 79.51 | Accuracy (%) | 63.6 |
| Nejumi 4 - ALT - Controllability | 88.03 | Score (%) | 59.7 |
| Nejumi 4 - jaster (0-shot) | 80.59 | Score (%) | 48.5 |
| Nejumi 4 - GLP - Syntactic Analysis | 81.33 | Score (%) | 48.1 |
| Nejumi 4 - ALT - Robustness | 87.5 | Score (%) | 44.8 |
Interactive version: theaggregate.ai/model?slug=llm-jp-4-1-33b-thinking · How It Works · Data refreshed daily, snapshot 2026-10-18.