llm-jp-4.1-32B-a3B: benchmark results
Provider: National Institute of Informatics. Released 2026-09-28. Access: Open.
Aggregate Rating 1628 ± 15, rank #360 of 1647 rated models, from 136 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM v2 - M-IFEval-Ja | 56.4 | Prompt-Level Strict Accuracy (%) | 100 |
| LLM-jp4.1 Release Swallow - MIFEval JA | 0.8 | Score (0–1) | 98.1 |
| Nejumi 4 - jaster (2-shot) - MAWPS | 99 | Exact match (%) | 96.8 |
| Nejumi 4 - BFCL - Irrelevance Detection | 93.33 | Accuracy (%) | 91.5 |
| LLM-jp4.1 Release Swallow - JEMHopQA | 0.65 | Score (0–1) | 88.5 |
| Open Japanese LLM v2 - JCoLA Out-of-Domain | 81.35 | Accuracy (%) | 86.4 |
| LLM-jp4.1 Release Swallow - AIME 2026 @32 | 0.93 | Pass@32 (0–1) | 84.6 |
| LLM-jp4.1 Release Swallow - Polymath JA top | 0.18 | Score (0–1) | 84.6 |
| LLM-jp4.1 Release Swallow - AIME 2024 @1 | 0.64 | Pass@1 (0–1) | 82.7 |
| LLM-jp4.1 Release Swallow - GPQA Diamond @1 | 0.6 | Pass@1 (0–1) | 82.7 |
| LLM-jp4.1 Release Swallow - JHumanEval @1 | 0.91 | Pass@1 (0–1) | 82.7 |
| LLM-jp4.1 Release Swallow - MCLM Math 100 @4 | 0.98 | Pass@4 (0–1) | 82.7 |
Interactive version: theaggregate.ai/model?slug=llm-jp-4-1-32b-a3b · How It Works · Data refreshed daily, snapshot 2026-10-17.