suzume-llama-3-8B-multilingual-orpo-borda-full: benchmark results
Lightblue's Suzume ORPO (borda-full), an ORPO fine-tune of their multilingual Llama 3 8B Suzume model on the full Mitsu preference dataset (all prompts, not filtered by ranking consistency). Provider: Lightblue. Released 2024-04-25. Access: Open.
Aggregate Rating 1434 ± 17, rank #1081 of 1656 rated models, from 29 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-IFEval | 46.3 | Strict accuracy, prompt/instruction mean (%) | 91 |
| Open Korean LLM - KorNAT-Harmless | 67 | Normalized accuracy (%) | 87.5 |
| Open Korean LLM - KorNAT-Helpful | 50.18 | Normalized accuracy (%) | 84.3 |
| Open Korean LLM - KorNAT-CKA | 29.96 | Normalized accuracy (%) | 81.6 |
| Open Korean LLM Leaderboard | 41.04 | Average Score (%) | 80.3 |
| Open Ko-LLM Leaderboard | 41.05 | Average (%) | 80.2 |
| Open Korean LLM - Ko-EQ-Bench | 30.11 | EQ-Bench score | 78.8 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 41.09 | Exact match, flexible extract (%) | 75.6 |
| Open LLM Leaderboard - IFEval | 58.17 | Score | 70 |
| Open Portuguese LLM - FaQuAD NLI | 72.64 | Macro F1 (%) | 66.4 |
| Open Portuguese LLM - ASSIN2 RTE | 91.09 | Macro F1 (%) | 63.6 |
| Open Chinese LLM - WinoGrande | 63.93 | Accuracy (%) | 62.4 |
Interactive version: theaggregate.ai/model?slug=suzume-llama-3-8b-multilingual-orpo-borda-full · How It Works · Data refreshed daily, snapshot 2026-10-20.