llm-jp-4-32B-a3B (Thinking): benchmark results
Provider: Other. Released 2026-03-16. Access: Open.
Unified ELO 1562 ± 1, rank #887 of 3078 rated models, from 139 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM v2 - JCulture-MCQ | 55.22 | Set F1 (%) | 100 |
| Open Japanese LLM v2 - JFinQA | 67.5 | Accuracy (%) | 100 |
| Nejumi 4 - BFCL - Irrelevance Detection | 93.33 | Accuracy (%) | 92.3 |
| Nejumi 4 - MT-Bench (Japanese) - Humanities | 100 | Judge rating (1-10, x10) | 90.4 |
| Nejumi 4 - MT-Bench (Japanese) - Math | 100 | Judge rating (1-10, x10) | 89 |
| Open Japanese LLM v2 - JCoLA Out-of-Domain | 80.12 | Accuracy (%) | 85.7 |
| Open Japanese LLM v2 - M-IFEval-Ja | 44.77 | Prompt-Level Strict Accuracy (%) | 83.3 |
| Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain) | 87 | Exact match (%) | 81.6 |
| Swallow - Japanese MT-Bench - Extraction | 75 | Judge Score (normalized, %) | 80.6 |
| Swallow - English MT-Bench - Extraction | 80.5 | Judge Score (normalized, %) | 76.1 |
| Nejumi 4 - Toxicity - Violation Categories | 47.62 | Criteria met (%) | 75.7 |
| Swallow - Japanese MT-Bench - Humanities | 68.6 | Judge Score (normalized, %) | 74.6 |
Interactive version: theaggregate.ai/model?slug=llm-jp-4-32b-a3b-thinking · How It Works · Data refreshed daily, snapshot 2026-09-19.