suzume-llama-3-8B-multilingual-orpo-borda-top25 — benchmark results
Lightblue's ORPO-tuned Suzume Llama 3 8B, preference-trained on the top 25% most consistently ranked responses from its Mitsu dataset. Provider: Lightblue. Released 2024-04-26. Access: Open.
Unified ELO 1405 ± 19, rank #1200 of 1776 rated models, from 15 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM Leaderboard | 530.94 | Average Score (%) | 90.3 |
| Open LLM Leaderboard - IFEval | 66.37 | Score | 78.4 |
| Open Chinese LLM - GSM8K | 48.9 | Accuracy (%) | 63.4 |
| Open LLM Leaderboard - MMLU-Pro | 29.83 | Score | 59.2 |
| Open Chinese LLM - WinoGrande | 63.54 | Accuracy (%) | 58.5 |
| Open Chinese LLM - CMMLU | 54.27 | Accuracy (%) | 55.3 |
| Open LLM Leaderboard - MATH Level 5 | 10.42 | Score | 48.7 |
| Open Chinese LLM Leaderboard | 56.28 | Average Score (%) | 45.3 |
| Open Chinese LLM - C-Eval Semantic | 67.88 | Accuracy (%) | 44.5 |
| Open LLM Leaderboard - BBH | 27.67 | Score | 44.2 |
| Open Chinese LLM - HellaSwag | 58.82 | Accuracy (%) | 40.7 |
| Open Chinese LLM - ARC Challenge | 48.72 | Accuracy (%) | 36.1 |
Interactive version: theaggregate.ai/model?slug=suzume-llama-3-8b-multilingual-orpo-borda-top25 · How the rankings work · Data refreshed daily, snapshot 2026-07-22.