Configurable-Hermes-2-Pro-Llama-3-8B — benchmark results
vicgalle's configurable-safety-tuning (CST) research fine-tune of Hermes 2 Pro Llama 3 8B, letting the system prompt set safety behavior at inference. Provider: Other. Released 2024-05-02. Access: Open.
Unified ELO 1424 ± 10, rank #1111 of 1776 rated models, from 21 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM Leaderboard | 487.36 | Average Score (%) | 86.3 |
| Open CoT - LSAT Logical Reasoning | 15.69 | CoT Gain (%) | 75.2 |
| Open Chinese LLM - TruthfulQA MC | 56.7 | Accuracy (%) | 71.5 |
| Open LLM Leaderboard - IFEval | 57.63 | Score | 69.5 |
| Open CoT - LogiQA 2 | 9.99 | CoT Gain (%) | 69.1 |
| Open CoT - LSAT Reading Comprehension | 15.24 | CoT Gain (%) | 67.9 |
| Open CoT Leaderboard | 9.93 | Average CoT Gain (%) | 67.9 |
| Open CoT - LogiQA | 5.27 | CoT Gain (%) | 65.6 |
| Open LLM Leaderboard - BBH | 30.51 | Score | 53.7 |
| Open LLM Leaderboard - GPQA | 6.26 | Score | 52.5 |
| Open LLM Leaderboard - MuSR | 10.06 | Score | 50 |
| Open Chinese LLM - GSM8K | 45.64 | Accuracy (%) | 49.6 |
Interactive version: theaggregate.ai/model?slug=configurable-hermes-2-pro-llama-3-8b · How the rankings work · Data refreshed daily, snapshot 2026-07-22.