Configurable-Hermes-2-Pro-Llama-3-8B: benchmark results
vicgalle's configurable-safety-tuning (CST) research fine-tune of Hermes 2 Pro Llama 3 8B, letting the system prompt set safety behavior at inference. Provider: Other. Released 2024-05-02. Access: Open.
Unified ELO 1467 ± 1, rank #861 of 1392 rated models, from 21 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM Leaderboard | 46.04 | Average Score (%) | 92.4 |
| Open CoT - LSAT Logical Reasoning | 15.69 | CoT Gain (%) | 75.2 |
| Open Chinese LLM - TruthfulQA MC | 56.7 | Accuracy (%) | 71.5 |
| Open LLM Leaderboard - IFEval | 57.63 | Score | 69.5 |
| Open CoT - LogiQA 2 | 9.99 | CoT Gain (%) | 69.1 |
| Open CoT - LSAT Reading Comprehension | 15.24 | CoT Gain (%) | 67.9 |
| Open CoT Leaderboard | 9.93 | Average CoT Gain (%) | 67.9 |
| Open CoT - LogiQA | 5.27 | CoT Gain (%) | 65.6 |
| Open LLM Leaderboard - BBH | 30.51 | Score | 53.6 |
| Open LLM Leaderboard - GPQA | 6.26 | Score | 52.6 |
| Open LLM Leaderboard - MuSR | 10.06 | Score | 50 |
| Open Chinese LLM - GSM8K | 45.64 | Accuracy (%) | 49.6 |
Interactive version: theaggregate.ai/model?slug=configurable-hermes-2-pro-llama-3-8b · How It Works · Data refreshed daily, snapshot 2026-09-05.