Open Japanese LLM - Xlsum JA Bert Score JA F1 Leaderboard — The Aggregate

Open Japanese LLM - Xlsum JA Bert Score JA F1 — leaderboard

Metric: Score (%). Source: huggingface.co. 418 models tracked.

Top models

#	Model	Score
1	aya-23-35B	72.12
2	aya-expanse-32B	71.67
3	Gemma 2 27B (IT)	71.62
4	Qwen 1.5 32B	71.47
5	Gemma 2 9B (IT)	71.14
6	Qwen 2 57B A14B	70.97
7	Qwen 2.5 32B	70.91
8	aya-23-8B	70.72
9	Phi-3.5-mini-instruct	70.67
10	Qwentile2.5-32B-Instruct	70.6
11	Qwen 2.5 14B	70.4
12	Qwen 1.5 32B Chat	70.34
13	QwQ 32B-Preview	70.31
14	Qwen 2 7B	70.26
15	Mistral Nemo Instruct (2407)	70.2

Interactive version: theaggregate.ai/benchmark?slug=open-japanese-llm-xlsum-ja-bert-score-ja-f1 · How the rankings work · Data refreshed daily, snapshot 2026-07-22.