Hermes-2-Theta-Llama-3-8B: benchmark results
Nous Research's Hermes 2 Theta, a merge of Hermes 2 Pro with Llama 3 8B Instruct (with Arcee's Charles Goddard) followed by further RLHF. Provider: Nous Research. Released 2024-05-05. Access: Open.
Aggregate Rating 1440 ± 15, rank #1047 of 1656 rated models, from 50 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open PL LLM - KLEJ NER (multiple choice, 5-shot) | 56.56 | Accuracy (%) | 88.4 |
| Open PL LLM - PolEmo2-IN (multiple choice, 5-shot) | 84.49 | Accuracy (%) | 84.4 |
| Open LLM Leaderboard - IFEval | 65.18 | Score | 77.5 |
| Open Portuguese LLM - FaQuAD NLI | 76.23 | Macro F1 (%) | 75.8 |
| Open Portuguese LLM - ENEM | 69.98 | Accuracy (%) | 74.6 |
| Open PL LLM - DYK (generative, 5-shot) | 64.55 | Binary F1 (%) | 73.9 |
| Open Portuguese LLM - ASSIN2 RTE | 92.07 | Macro F1 (%) | 73.9 |
| Open PL LLM - PSC (generative, 5-shot) | 95.87 | Binary F1 (%) | 73.2 |
| Open Chinese LLM - GSM8K | 53.37 | Accuracy (%) | 72.3 |
| Open PL LLM Leaderboard | 55.24 | Average Score (%) | 68.4 |
| Open PL LLM - Multiple Choice | 50.87 | Average Multiple-Choice Score (%) | 68.2 |
| Open Portuguese LLM - BLUEX | 57.02 | Accuracy (%) | 68.1 |
Interactive version: theaggregate.ai/model?slug=hermes-2-theta-llama-3-8b · How It Works · Data refreshed daily, snapshot 2026-10-20.