Meta-Llama-3-70B-Instruct-abliterated-v3.5: benchmark results
failspy's abliterated Llama 3 70B Instruct (v3.5) with the refusal direction orthogonalized out of the weights, modifying only a single layer. Provider: failspy. Released 2024-05-28. Access: Open.
Aggregate Rating 1510 ± 43, rank #2196 of 10916 rated models, from 14 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard - IFEval | 77.47 | Score | 93.4 |
| Open LLM Leaderboard - MMLU-Pro | 44.52 | Score | 82.5 |
| Open LLM Leaderboard - BBH | 57.47 | Score | 78.8 |
| Open Chinese LLM - CMMLU | 64.66 | Accuracy (%) | 76.6 |
| Open Chinese LLM - C-Eval Semantic | 82.45 | Accuracy (%) | 73.1 |
| Open Chinese LLM Leaderboard | 60.89 | Official Nonzero-Task Mean (%) | 71 |
| Open Chinese LLM - GSM8K | 49.58 | Accuracy (%) | 65.7 |
| Open Chinese LLM - ARC Challenge | 53.67 | Accuracy (%) | 58.9 |
| Open LLM Leaderboard - MATH Level 5 | 12.84 | Score | 56.9 |
| Open LLM Leaderboard - GPQA | 29.7 | Score | 52.5 |
| Open Chinese LLM - HellaSwag | 59.83 | Accuracy (%) | 48.2 |
| Open LLM Leaderboard - MuSR | 39.82 | Score | 40.7 |
Interactive version: theaggregate.ai/model?slug=meta-llama-3-70b-instruct-abliterated-v3-5 · How It Works · Data refreshed daily, snapshot 2026-11-08.