suzume-llama-3-8B-multilingual-orpo-borda-top75: benchmark results
Lightblue's Suzume ORPO (borda-top75), an ORPO fine-tune of their multilingual Llama 3 8B Suzume model on the top 75% most consistently ranked responses in the Mitsu preference dataset. Provider: Lightblue. Released 2024-04-26. Access: Open.
Aggregate Rating 1460 ± 18, rank #939 of 1656 rated models, from 29 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-IFEval | 50.88 | Strict accuracy, prompt/instruction mean (%) | 94.2 |
| Open Ko-LLM Leaderboard | 45.5 | Average (%) | 92 |
| Open Korean LLM Leaderboard | 45.5 | Average Score (%) | 92 |
| Open Korean LLM - KorNAT-CKA | 33.21 | Normalized accuracy (%) | 91.1 |
| Open Korean LLM - Ko-EQ-Bench | 37.95 | EQ-Bench score | 88.6 |
| Open Korean LLM - KorNAT-Harmless | 66.96 | Normalized accuracy (%) | 87.4 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 47.01 | Exact match, flexible extract (%) | 81.1 |
| Open LLM Leaderboard - IFEval | 66.87 | Score | 78.9 |
| Open Portuguese LLM - BLUEX | 57.72 | Accuracy (%) | 70.7 |
| Open Portuguese LLM - OAB Exams | 48.2 | Accuracy (%) | 68.7 |
| Open Portuguese LLM - ENEM | 67.32 | Accuracy (%) | 66.1 |
| Open LLM Leaderboard - MMLU-Pro | 37.69 | Score | 63.7 |
Interactive version: theaggregate.ai/model?slug=suzume-llama-3-8b-multilingual-orpo-borda-top75 · How It Works · Data refreshed daily, snapshot 2026-10-20.