HardMTBench - English to Chinese xCOMET: leaderboard

HardMTBench (2026) is a difficulty-aware Chinese-English translation test of 10,000 pairs from 12 knowledge-intensive domains, selected for domain-knowledge density and translation difficulty. This board scores the English-to-Chinese direction with the reference-based xCOMET-XXL metric, in percent; higher is better.

Metric: xCOMET-XXL score (%) of the English-to-Chinese translations, a reference-based learned metric (0-1) times 100, on HardMTBench, 10,000 domain-balanced, difficulty-selected Chinese-English pairs from 12 knowledge-intensive domains, each translated in both directions under one unified protocol; higher is better. Source: arxiv.org. Saturation forecast: Around October 2027. 16 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Hy-MT2-30B-A3B75.87
2Hy-MT2-7B75.35
3Gemini 3.1 Pro (Preview)74.15#99
4GPT-5.573.44#54
5Qwen-MT-Plus72.63
6Qwen 3.6 35B A3B72.57#481
7Qwen 3.5 397B A17B72.36#264
8Gemma 4 31B (IT)Thinking72.1#350
9gemma-4-26B-A4B-itThinking71.81
10DeepSeek V4 ProThinking71.66#196
11Gemma 4 26B A4B (IT)71.09#556
12Hy-MT2-1.8B70.72
13Gemma 4 E4B (IT)Thinking67.35
14gemma-4-E4B-it67.04#1539
15Gemma 4 E2B (IT)Thinking65.72#2451

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=hardmtbench-english-to-chinese-xcomet · How It Works · Data refreshed daily, snapshot 2026-11-21.