HardMTBench - Chinese to English xCOMET: leaderboard
HardMTBench (2026) is a difficulty-aware Chinese-English translation test of 10,000 pairs from 12 knowledge-intensive domains, selected for domain-knowledge density and translation difficulty. This board scores the Chinese-to-English direction with the reference-based xCOMET-XXL metric, in percent; higher is better.
Metric: xCOMET-XXL score (%) of the Chinese-to-English translations, a reference-based learned metric (0-1) times 100, on HardMTBench, 10,000 domain-balanced, difficulty-selected Chinese-English pairs from 12 knowledge-intensive domains, each translated in both directions under one unified protocol; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 16 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Hy-MT2-30B-A3B | 69.43 | ||
| 2 | Hy-MT2-7B | 67.51 | ||
| 3 | Qwen 3.5 397B A17B | Non-reasoning | 65.77 | #270 |
| 4 | GPT-5.5 | Non-reasoning | 65.76 | #56 |
| 5 | Gemma 4 31B (IT) | Thinking | 65.7 | #354 |
| 6 | Qwen 3.6 35B A3B | 65.62 | #485 | |
| 7 | Qwen-MT-Plus | 65.44 | ||
| 8 | DeepSeek V4 Pro | Non-reasoning | 65.09 | #201 |
| 9 | gemma-4-26B-A4B-it | Thinking | 64.96 | |
| 10 | Gemini 3.1 Pro (Preview) | 64.45 | #102 | |
| 11 | Gemma 4 26B A4B (IT) | 63.81 | #563 | |
| 12 | Hy-MT2-1.8B | 61.94 | ||
| 13 | Gemma 4 E4B (IT) | Thinking | 60.39 | |
| 14 | gemma-4-E4B-it | 59.23 | #1577 | |
| 15 | Gemma 4 E2B (IT) | Thinking | 57.4 | #2532 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=hardmtbench-chinese-to-english-xcomet · How It Works · Data refreshed daily, snapshot 2026-11-22.