suzume-llama-3-8B-multilingual — benchmark results
Lightblue's Suzume, a Llama 3 8B Instruct fine-tune on ~90K multilingual conversations to add non-English chat ability to Llama 3. Provider: Lightblue. Released 2024-05-01. Access: Open.
Unified ELO 1401 ± 11, rank #1219 of 1776 rated models, from 36 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM Leaderboard | 521.74 | Average Score (%) | 89.6 |
| Open Medical LLM - MMLU College Biology | 79.17 | Accuracy (%) | 80.7 |
| Open LLM Leaderboard - IFEval | 66.78 | Score | 78.8 |
| Open Medical LLM - MMLU College Medicine | 65.9 | Accuracy (%) | 75.3 |
| Open Medical LLM - MMLU Anatomy | 65.93 | Accuracy (%) | 73.9 |
| Open Medical LLM - MMLU Medical Genetics | 81 | Accuracy (%) | 71.6 |
| Open Medical LLM - MedQA (USMLE) | 60.09 | Accuracy (%) | 70.7 |
| Open Medical LLM - MedMCQA | 56.61 | Accuracy (%) | 69.3 |
| Open Medical LLM | 69.3 | Average Accuracy (%) | 67.3 |
| Open Medical LLM - MMLU Clinical Knowledge | 72.45 | Accuracy (%) | 62.5 |
| Open Chinese LLM - WinoGrande | 63.54 | Accuracy (%) | 58.5 |
| Open Medical LLM - MMLU Professional Medicine | 70.96 | Accuracy (%) | 58.2 |
Interactive version: theaggregate.ai/model?slug=suzume-llama-3-8b-multilingual · How the rankings work · Data refreshed daily, snapshot 2026-07-22.