Kurakurai French LLM - MBPP: leaderboard

Evaluates small French-language models on the Kurakurai French LLM leaderboard, a static board of French translations and French subsets of standard benchmarks, filtered by size class up to about 4B parameters. Scores are percentages averaged over ten sampling seeds, decoded at temperature 0.6, top_p 0.95, top_k 20, with thinking disabled and 16k context. This series is the board's MBPP column, which upstream documents as MBPP+ (French) code generation. These are French scores, comparable to each other but not to published English numbers for the same models. This series reports Kurakurai French LLM - MBPP using pass@1 (%).

Metric: pass@1 (%). Source: huggingface.co. Saturation forecast: Around February 2027. 42 models tracked.

Top models

#ModelScoreOverall rank
1Qwen 3 4B 2507 Instruct76.03#769
2Gemma 4 E2B (IT)68.68#780
3Granite 4.1 3B68.07#904
4Qwen 3.5 4B67.75#515
5granite-4.0-h-micro66.32#1087
6Gemma 3 4B (IT)62.04#996
7Qwen 3 1.7B57.33#1183
8SmolLM3-3B56.14#1238
9LFM2 2.6B54.42#1314
10LFM2.5-1.2B Instruct48.65#1452
11Llama 3.2 3B Instruct44.26#1328
12Gemma 3 1B (IT)43.86#1482
13Qwen 3.5 2B42.12#1115
14Phi-4 Mini Instruct39.18#1232
15Qwen 3 0.6B37.41#1449

No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.

Interactive version: theaggregate.ai/benchmark?slug=kurakurai-french-llm-mbpp · How It Works · Data refreshed daily, snapshot 2026-11-06.