Llama-3-8B-Instruct-abliterated-dpomix: benchmark results
mlabonne's Llama-3-8B-Instruct-abliterated-dpomix, a DPO fine-tune of an abliterated (refusal-removed) Llama 3 8B Instruct on the orpo-dpo-mix-40k dataset, aimed at recovering capability lost during abliteration. Provider: mlabonne. Released 2024-06-01. Access: Open.
Aggregate Rating 1437 ± 16, rank #1046 of 1635 rated models, from 31 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Medical LLM - MMLU Medical Genetics | 85 | Accuracy (%) | 95.8 |
| Open Korean LLM - Ko-IFEval | 47.55 | Strict accuracy, prompt/instruction mean (%) | 91.9 |
| Open LLM Leaderboard v1 - GSM8K | 69.75 | Accuracy (%) (5-shot) | 90.3 |
| Open LLM Leaderboard v1 - MMLU | 67 | Accuracy (%) (5-shot) | 88.6 |
| Open Medical LLM - PubMedQA | 76.2 | Accuracy (%) | 82.6 |
| Open Korean LLM - Ko-GPQA-Diamond | 25.76 | Normalized accuracy (%) | 81.3 |
| Open Korean LLM Leaderboard | 40.93 | Average Score (%) | 79.8 |
| Open Medical LLM - MMLU Professional Medicine | 74.63 | Accuracy (%) | 79.2 |
| Open Portuguese LLM - OAB Exams | 50.98 | Accuracy (%) | 78.8 |
| Open Portuguese LLM - ENEM | 70.54 | Accuracy (%) | 76.7 |
| Open Medical LLM - MedQA (USMLE) | 60.64 | Accuracy (%) | 75 |
| Open Medical LLM - MMLU College Medicine | 65.9 | Accuracy (%) | 74.7 |
Interactive version: theaggregate.ai/model?slug=llama-3-8b-instruct-abliterated-dpomix · How It Works · Data refreshed daily, snapshot 2026-11-05.