Llama 3 8B Orpo V0.1: benchmark results
A community ORPO fine-tune of Meta's Llama 3 8B by dfurman, trained on 4,000 samples from the orpo-dpo-mix-40k dataset with an 8K context and ChatML formatting. Provider: dfurman. Released 2024-04-26. Access: Open.
Aggregate Rating 1363 ± 15, rank #1358 of 1656 rated models, from 35 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard v1 - MMLU | 66.59 | Accuracy (%) (5-shot) | 87.7 |
| Open Korean LLM - Ko-Winogrande | 63.14 | Accuracy (%) | 72.6 |
| Open LLM Leaderboard v1 - WinoGrande | 79.01 | Accuracy (%) (5-shot) | 66.7 |
| Open Chinese LLM - WinoGrande | 64.33 | Accuracy (%) | 66.1 |
| Open Korean LLM - Ko-GPQA-Diamond | 24.24 | Normalized accuracy (%) | 62.4 |
| Open LLM Leaderboard v1 - GSM8K | 48.75 | Accuracy (%) (5-shot) | 61.8 |
| Open Korean LLM - KorNAT-Helpful | 47.78 | Normalized accuracy (%) | 60.3 |
| Open Korean LLM - Ko-IFEval | 30.45 | Strict accuracy, prompt/instruction mean (%) | 56.2 |
| Open LLM Leaderboard v1 - HellaSwag | 82.56 | Normalized accuracy (%) (10-shot) | 52.3 |
| Open Chinese LLM - CMMLU | 53.37 | Accuracy (%) | 49.7 |
| Open LLM Leaderboard v1 - ARC Challenge | 60.67 | Normalized accuracy (%) (25-shot) | 48.9 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 50.47 | MC2 (%) (0-shot) | 48.4 |
Interactive version: theaggregate.ai/model?slug=llama-3-8b-orpo-v0-1 · How It Works · Data refreshed daily, snapshot 2026-10-20.