DeepHermes-3-Mistral-24B-Preview: benchmark results
Nous Research's DeepHermes 3 preview on Mistral Small 24B, toggling between plain chat and long R1-distilled reasoning via system prompt (March 2025). Provider: Nous Research. Released 2025-03-02. Access: Open.
Aggregate Rating 1473 ± 35, rank #2782 of 10916 rated models, from 12 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard - GPQA | 37 | Score | 92.5 |
| Open LLM Leaderboard - BBH | 64.88 | Score | 90.7 |
| Open LLM Leaderboard - MMLU-Pro | 45.9 | Score | 84.8 |
| Open LLM Leaderboard - MuSR | 45.03 | Score | 83.6 |
| Open LLM Leaderboard - MATH Level 5 | 25.76 | Score | 78.6 |
| Open LLM Leaderboard - IFEval | 45.36 | Score | 50.1 |
| AA MMLU-Pro | 58.01 | Accuracy (%) | 21.6 |
| AA LiveCodeBench | 19.47 | Pass@1 (%) | 20.6 |
| AA GPQA Diamond | 38.18 | Accuracy (%) | 13.8 |
| SnakeBench | 13.3 | TrueSkill Rating | 11.1 |
| AA Humanity's Last Exam | 3.76 | Accuracy (%) | 8.8 |
| AA MATH-500 | 59.47 | Accuracy (%) | 6.7 |
Interactive version: theaggregate.ai/model?slug=deephermes-3-mistral-24b-preview · How It Works · Data refreshed daily, snapshot 2026-11-08.