Benchmarking Large Language Models for Graphem: leaderboard
Grapheme-to-phoneme (G2P) conversion is essential for controllable and robust text-to-speech, and large language models (LLMs), with broad linguistic knowledge, offer a promising approach. We benchmarked over 30 LLMs on Japanese G2P, comparing them with conventional morphological analyzers on 3000 manually annotated sentences. We evaluated two prompting strategies: a parse mode, where the LLM perf
Metric: Direct (self-reported). Source: benchmarklist.com. Saturation forecast: Estimated already saturated. 37 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 0.53 | #58 |
| 2 | Claude Opus 4.6 | 0.74 | #63 |
| 3 | GPT-5.2 | 1.05 | #110 |
| 4 | Gemini 2.5 Flash | 1.17 | #245 |
| 5 | Claude Sonnet 4.6 | 1.28 | #90 |
| 6 | Gemini 3 Flash | 1.61 | #99 |
| 7 | GPT-OSS-120B | 3.45 | #341 |
| 8 | Kimi K2.5 | 4.91 | #144 |
| 9 | GLM-5 | 4.96 | #141 |
| 10 | GLM-4.7 | 5.87 | #192 |
| 11 | Qwen 3.5 397B A17B | 6.2 | #146 |
| 12 | Qwen 3.5 27B | 14.61 | #187 |
| 13 | GPT-OSS-20B | 14.71 | #518 |
| 14 | Gemma 3 27B | 15.16 | #607 |
| 15 | Qwen 3.5 122B A10B | 16.8 | #173 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=benchmarking-large-language-models-for-graphem · How It Works · Data refreshed daily, snapshot 2026-10-20.