llama-3-70B-Instruct-abliterated: benchmark results
failspy's abliterated Llama 3 70B Instruct with the refusal direction orthogonalized out of the weights. Provider: Meta. Released 2024-05-07. Access: Open.
Aggregate Rating 1586 ± 46, rank #1174 of 10916 rated models, from 14 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Chinese LLM - GSM8K | 71.34 | Accuracy (%) | 97.9 |
| Open LLM Leaderboard - IFEval | 80.23 | Score | 97.6 |
| Open Chinese LLM - CMMLU | 71.56 | Accuracy (%) | 91.7 |
| Open LLM Leaderboard - BBH | 64.65 | Score | 90.2 |
| Open LLM Leaderboard - MMLU-Pro | 51.45 | Score | 89.1 |
| Open Chinese LLM Leaderboard | 67.35 | Official Nonzero-Task Mean (%) | 86.7 |
| Open Chinese LLM - ARC Challenge | 59.73 | Accuracy (%) | 86.2 |
| Open Chinese LLM - C-Eval Semantic | 86.76 | Accuracy (%) | 81.8 |
| Open LLM Leaderboard - MATH Level 5 | 24.32 | Score | 77.5 |
| Open Chinese LLM - HellaSwag | 63.32 | Accuracy (%) | 72.5 |
| Open LLM Leaderboard - MuSR | 41.28 | Score | 53 |
| Open LLM Leaderboard - GPQA | 28.94 | Score | 44 |
Interactive version: theaggregate.ai/model?slug=llama-3-70b-instruct-abliterated · How It Works · Data refreshed daily, snapshot 2026-11-08.