HardMTBench - English to Chinese xCOMET: leaderboard
HardMTBench (2026) is a difficulty-aware Chinese-English translation test of 10,000 pairs from 12 knowledge-intensive domains, selected for domain-knowledge density and translation difficulty. This board scores the English-to-Chinese direction with the reference-based xCOMET-XXL metric, in percent; higher is better.
Metric: xCOMET-XXL score (%) of the English-to-Chinese translations, a reference-based learned metric (0-1) times 100, on HardMTBench, 10,000 domain-balanced, difficulty-selected Chinese-English pairs from 12 knowledge-intensive domains, each translated in both directions under one unified protocol; higher is better. Source: arxiv.org. Saturation forecast: Around October 2027. 16 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Hy-MT2-30B-A3B | 75.87 | ||
| 2 | Hy-MT2-7B | 75.35 | ||
| 3 | Gemini 3.1 Pro (Preview) | 74.15 | #99 | |
| 4 | GPT-5.5 | 73.44 | #54 | |
| 5 | Qwen-MT-Plus | 72.63 | ||
| 6 | Qwen 3.6 35B A3B | 72.57 | #481 | |
| 7 | Qwen 3.5 397B A17B | 72.36 | #264 | |
| 8 | Gemma 4 31B (IT) | Thinking | 72.1 | #350 |
| 9 | gemma-4-26B-A4B-it | Thinking | 71.81 | |
| 10 | DeepSeek V4 Pro | Thinking | 71.66 | #196 |
| 11 | Gemma 4 26B A4B (IT) | 71.09 | #556 | |
| 12 | Hy-MT2-1.8B | 70.72 | ||
| 13 | Gemma 4 E4B (IT) | Thinking | 67.35 | |
| 14 | gemma-4-E4B-it | 67.04 | #1539 | |
| 15 | Gemma 4 E2B (IT) | Thinking | 65.72 | #2451 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=hardmtbench-english-to-chinese-xcomet · How It Works · Data refreshed daily, snapshot 2026-11-21.