English-Hindi Gender Recoverability: leaderboard
English-to-Hindi translation of 37,345 sentences in twelve categories, from explicit gender and Winograd-style coreference to name, profession and gender-neutral sentences (Savi et al., 2026). A translation counts as correct when a gender cue that the English source expresses stays recoverable in the Hindi output, or stays neutral when the source is ambiguous, as judged by a rule-based Hindi classifier with an LLM fallback; accuracy in percent, higher is better. Ergative and honorific constructions are the main way Hindi output erases the cue.
Metric: Gender-recoverability accuracy (%; all 37,345 English-to-Hindi instances in twelve categories, unreranked MT output, rule-based Hindi classifier with LLM fallback). Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4o Mini | 57.18 | #1627 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=english-hindi-gender-recoverability · How It Works · Data refreshed daily, snapshot 2026-11-12.