Open Japanese LLM - Niilc Char F1 Leaderboard — The Aggregate

Open Japanese LLM - Niilc Char F1: leaderboard

Metric: Score (%). Source: huggingface.co. 418 models tracked.

Top models

#	Model	Score
1	Llama-3.3-Swallow-70B-Instruct-v0.4	67.39
2	aya-expanse-32B	61.79
3	ABEJA-Qwen2.5-32B-Japanese-v1.0	60.93
4	Llama 3.1 70B	60.2
5	Llama 3 70B	59.91
6	Llama 3.1 Nemotron 70B Instruct	58.83
7	Llama 3.1 70B Instruct	57.73
8	Llama 3.3 70B Instruct	56.25
9	Gemma 2 27B (IT)	56.07
10	Llama 3 70B Instruct	55.44
11	aya-23-35B	55.01
12	Qwen 2.5 32B	54.64
13	Llama 2 70B Base	54.43
14	tempmotacilla-cinerea-0308	53.66
15	Qwen 2.5 14B	53.41

Interactive version: theaggregate.ai/benchmark?slug=open-japanese-llm-niilc-char-f1 · How It Works · Data refreshed daily, snapshot 2026-09-05.