llm-jp-4-33B: benchmark results
Provider: National Institute of Informatics. Released 2026-08-18. Access: Open.
Aggregate Rating 1639 ± 16, rank #328 of 1647 rated models, from 131 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| LLM-jp4.1 Release Swallow - JEMHopQA | 0.68 | Score (0–1) | 100 |
| LLM-jp4.1 Release Swallow - JAM-CQA | 0.58 | Score (0–1) | 94.2 |
| LLM-jp4.1 Release Swallow - JHumanEval @10 | 0.96 | Pass@10 (0–1) | 94.2 |
| LLM-jp4.1 Release Swallow - WMT20 EN→JA | 0.25 | BLEU (source-normalized 0–1) | 92.3 |
| LLM-jp4.1 Release Swallow - WMT20 JA→EN | 0.21 | BLEU (source-normalized 0–1) | 92.3 |
| LLM-jp4.1 Release Swallow - JHumanEval @1 | 0.92 | Pass@1 (0–1) | 90.4 |
| LLM-jp4.1 Release Swallow - JMMLU | 0.85 | Score (0–1) | 90.4 |
| Nejumi 4 - MT-Bench (Japanese) - STEM | 100 | Judge rating (1-10, x10) | 90.4 |
| Nejumi 4 - MT-Bench (Japanese) - Math | 100 | Judge rating (1-10, x10) | 89.4 |
| LLM-jp4.1 Release Swallow - AIME 2024 @32 | 0.93 | Pass@32 (0–1) | 88.5 |
| LLM-jp4.1 Release Swallow - HumanEval+ @1 | 0.92 | Pass@1 (0–1) | 88.5 |
| LLM-jp4.1 Release Swallow - LiveCodeBench v6 @10 | 0.65 | Pass@10 (0–1) | 88.5 |
Interactive version: theaggregate.ai/model?slug=llm-jp-4-33b · How It Works · Data refreshed daily, snapshot 2026-10-17.