HardMTBench - Chinese to English xCOMET: leaderboard

HardMTBench (2026) is a difficulty-aware Chinese-English translation test of 10,000 pairs from 12 knowledge-intensive domains, selected for domain-knowledge density and translation difficulty. This board scores the Chinese-to-English direction with the reference-based xCOMET-XXL metric, in percent; higher is better.

Metric: xCOMET-XXL score (%) of the Chinese-to-English translations, a reference-based learned metric (0-1) times 100, on HardMTBench, 10,000 domain-balanced, difficulty-selected Chinese-English pairs from 12 knowledge-intensive domains, each translated in both directions under one unified protocol; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 16 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Hy-MT2-30B-A3B69.43
2Hy-MT2-7B67.51
3Qwen 3.5 397B A17BNon-reasoning65.77#270
4GPT-5.5Non-reasoning65.76#56
5Gemma 4 31B (IT)Thinking65.7#354
6Qwen 3.6 35B A3B65.62#485
7Qwen-MT-Plus65.44
8DeepSeek V4 ProNon-reasoning65.09#201
9gemma-4-26B-A4B-itThinking64.96
10Gemini 3.1 Pro (Preview)64.45#102
11Gemma 4 26B A4B (IT)63.81#563
12Hy-MT2-1.8B61.94
13Gemma 4 E4B (IT)Thinking60.39
14gemma-4-E4B-it59.23#1577
15Gemma 4 E2B (IT)Thinking57.4#2532

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=hardmtbench-chinese-to-english-xcomet · How It Works · Data refreshed daily, snapshot 2026-11-22.