Hermes-2-Pro-Llama-3-8B: benchmark results
Nous Research's Hermes 2 Pro on Llama 3 8B, tuned on cleaned OpenHermes 2.5 data for function calling and structured JSON output. Provider: Nous Research. Released 2024-04-30. Access: Open.
Aggregate Rating 1424 ± 17, rank #1127 of 1656 rated models, from 83 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard v1 - GSM8K | 67.93 | Accuracy (%) (5-shot) | 84.7 |
| Open PL LLM - PolEmo2-IN (multiple choice, 5-shot) | 84.35 | Accuracy (%) | 83.1 |
| SWE-Arena | 1002 | Elo Score | 81.9 |
| Open PL LLM - KLEJ NER (multiple choice, 5-shot) | 54.52 | Accuracy (%) | 80.8 |
| Open LLM Leaderboard v1 - MMLU | 64.8 | Accuracy (%) (5-shot) | 76.8 |
| Open Portuguese LLM - ASSIN2 RTE | 92.24 | Macro F1 (%) | 76.6 |
| Open PL LLM - PSC (generative, 5-shot) | 95.58 | Binary F1 (%) | 71.9 |
| Open Portuguese LLM - FaQuAD NLI | 74.87 | Macro F1 (%) | 71.7 |
| Open PL LLM - PSC (multiple choice, 5-shot) | 79.43 | Binary F1 (%) | 70.8 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 56.56 | MC2 (%) (0-shot) | 68.4 |
| Open Portuguese LLM - BLUEX | 57.02 | Accuracy (%) | 68.1 |
| Open Portuguese LLM - ENEM | 67.88 | Accuracy (%) | 68.1 |
Interactive version: theaggregate.ai/model?slug=hermes-2-pro-llama-3-8b · How It Works · Data refreshed daily, snapshot 2026-10-20.