Kurakurai French LLM - MMLU-Pro: leaderboard

Evaluates small French-language models on the Kurakurai French LLM leaderboard, a static board of French translations and French subsets of standard benchmarks, filtered by size class up to about 4B parameters. Scores are percentages averaged over ten sampling seeds, decoded at temperature 0.6, top_p 0.95, top_k 20, with thinking disabled and 16k context. This series is MMLU-Pro (French), advanced multi-task knowledge. These are French scores, comparable to each other but not to published English numbers for the same models. This series reports Kurakurai French LLM - MMLU-Pro using Accuracy (%).

Metric: Accuracy (%). Source: huggingface.co. Saturation forecast: Around February 2027. 42 models tracked.

Top models

#ModelScoreOverall rank
1Qwen 3.5 4B71.4#515
2Gemma 4 E2B (IT)56.4#780
3Qwen 3 4B 2507 Instruct56.3#769
4Qwen 3.5 2B47.2#1115
5LFM2 2.6B45.1#1314
6Granite 4.1 3B43.2#904
7Gemma 3 4B (IT)39.4#996
8Qwen 3 1.7B39.2#1183
9granite-4.0-h-micro38.45#1087
10LFM2.5-1.2B Instruct37.4#1452
11SmolLM3-3B36.9#1238
12Phi-4 Mini Instruct35.7#1232
13Tiny Aya Global29.4#1306
14Qwen 3.5 0.8B27.6#1411
15Luth-LFM2-1.2B26#1386

No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.

Interactive version: theaggregate.ai/benchmark?slug=kurakurai-french-llm-mmlu-pro · How It Works · Data refreshed daily, snapshot 2026-11-06.