llm-jp-4.1-32B-a3B: benchmark results

Provider: National Institute of Informatics. Released 2026-09-28. Access: Open.

Aggregate Rating 1627 ± 15, rank #367 of 1636 rated models, from 136 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
Open Japanese LLM v2 - M-IFEval-Ja56.4Prompt-Level Strict Accuracy (%)100
LLM-jp4.1 Release Swallow - MIFEval JA0.8Score (0–1)98.1
Nejumi 4 - jaster (2-shot) - MAWPS99Exact match (%)96.9
Nejumi 4 - BFCL - Irrelevance Detection93.33Accuracy (%)91.7
LLM-jp4.1 Release Swallow - JEMHopQA0.65Score (0–1)88.5
Open Japanese LLM v2 - JCoLA Out-of-Domain81.35Accuracy (%)87.5
LLM-jp4.1 Release Swallow - AIME 2026 @320.93Pass@32 (0–1)84.6
LLM-jp4.1 Release Swallow - Polymath JA top0.18Score (0–1)84.6
Nejumi 4 - jaster (0-shot) - CommonsenseMoralJA96Exact match (%)83
LLM-jp4.1 Release Swallow - AIME 2024 @10.64Pass@1 (0–1)82.7
LLM-jp4.1 Release Swallow - GPQA Diamond @10.6Pass@1 (0–1)82.7
LLM-jp4.1 Release Swallow - JHumanEval @10.91Pass@1 (0–1)82.7

Interactive version: theaggregate.ai/model?slug=llm-jp-4-1-32b-a3b · How It Works · Data refreshed daily, snapshot 2026-11-06.