RakutenAI-7B-chat: benchmark results
Rakuten's RakutenAI-7B-chat, a Mistral-7B-v0.1-based chat model for Japanese and English with vocabulary expanded from 32K to 48K tokens for Japanese efficiency. Provider: Rakuten. Released 2024-03-18. Access: Open.
Aggregate Rating 1416 ± 28, rank #1168 of 1656 rated models, from 70 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM - ALT E TO J Bleu JA | 13.07 | Score (%) | 73.3 |
| Open Japanese LLM - Xlsum JA Rouge1 | 28.49 | Score (%) | 71.6 |
| Enkrypt AI - Bias Risk | 75.71 | Risk Score | 71.3 |
| Open Japanese LLM - Xlsum JA RougeLsum | 23.91 | Score (%) | 69.7 |
| Open Japanese LLM - Jnli Exact Match | 85.09 | Score (%) | 69 |
| Open Japanese LLM - ALT E TO J Bert Score JA F1 | 86.24 | Score (%) | 66.1 |
| Open Japanese LLM - Xlsum JA Bert Score JA F1 | 69.61 | Score (%) | 64.9 |
| Open Japanese LLM - Xlsum JA Rouge2 | 10.18 | Score (%) | 62.6 |
| Open Japanese LLM - SUM | 10.17 | Score (%) | 62.5 |
| Open Japanese LLM - ALT J TO E Bleu EN | 16.85 | Score (%) | 60.7 |
| pfgen-bench - Completion Mode - Fluency | 0.59 | Fluency Score | 55.3 |
| pfgen-bench - Completion Mode - Truthfulness | 0.76 | Truthfulness Score | 54.8 |
Interactive version: theaggregate.ai/model?slug=rakutenai-7b-chat · How It Works · Data refreshed daily, snapshot 2026-10-20.