Nous-Hermes-2-SOLAR-10.7B: benchmark results
Nous Research's Nous-Hermes-2-SOLAR-10.7B, fine-tuned from SOLAR-10.7B-v1.0 on about 1,000,000 examples of mostly GPT-4-generated data, using the ChatML prompt format. Provider: Nous Research. Released 2024-01-01. Access: Open.
Aggregate Rating 1443 ± 13, rank #1032 of 1656 rated models, from 63 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open PL LLM - PSC (generative, 5-shot) | 97.99 | Binary F1 (%) | 91.4 |
| Open PL LLM - PoQuAD Open Book (generative, 5-shot) | 71.13 | Levenshtein Similarity (%) | 89.5 |
| Open Portuguese LLM - FaQuAD NLI | 80.09 | Macro F1 (%) | 89.3 |
| Open LLM Leaderboard v1 - GSM8K | 69.45 | Accuracy (%) (5-shot) | 89.2 |
| Open LLM Leaderboard v1 - MMLU | 66.3 | Accuracy (%) (5-shot) | 86.7 |
| Open LLM Leaderboard v1 - WinoGrande | 82.79 | Accuracy (%) (5-shot) | 85.6 |
| Open PL LLM - PolEmo2-OUT (multiple choice, 5-shot) | 76.92 | Accuracy (%) | 83.7 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 49.51 | Exact match, flexible extract (%) | 83.1 |
| Open Korean LLM - Ko-IFEval | 40.27 | Strict accuracy, prompt/instruction mean (%) | 82.6 |
| Open PL LLM - KLEJ NER (multiple choice, 5-shot) | 54.62 | Accuracy (%) | 81.9 |
| Open Ko-LLM Leaderboard | 40.92 | Average (%) | 79.5 |
| Open Korean LLM - KorNAT-Harmless | 65.11 | Normalized accuracy (%) | 79.2 |
Interactive version: theaggregate.ai/model?slug=nous-hermes-2-solar-10-7b · How It Works · Data refreshed daily, snapshot 2026-10-20.