Open Japanese LLM - Xlsum JA Bleu JA Leaderboard — The Aggregate

Open Japanese LLM - Xlsum JA Bleu JA — leaderboard

Metric: Score (%). Source: huggingface.co. 418 models tracked.

Top models

#	Model	Score
1	Llama 2 70B Chat (HF)	78.31
2	Llama 2 13B Base	75.69
3	DeepSeek-R1-Distill-Qwen-7B	75.15
4	Qwen 1.5 4B	62.09
5	Saka-1.5B	51.76
6	DeepSeek R1 Distill Qwen 1.5B	43
7	Llama 2 70B Base	35.45
8	Llama 3 8B Instruct	34.1
9	Llama 2 13B Chat Base	33.8
10	Qwen 2.5 Coder 7B Instruct	19.22
11	Qwen2.5-Math-7B-Instruct	14.76
12	aya-23-35B	4.15
13	aya-expanse-32B	3.66
14	Llama 3.1 70B Instruct	3.43
15	Llama 3.1 Nemotron 70B Instruct	3.41

Interactive version: theaggregate.ai/benchmark?slug=open-japanese-llm-xlsum-ja-bleu-ja · How the rankings work · Data refreshed daily, snapshot 2026-07-22.