Hermes-2-Pro-Mistral-7B Benchmark Results

Hermes-2-Pro-Mistral-7B — benchmark results

Provider: Nous Research. Released 2024-03-14. Access: Open.

Unified ELO 1347 ± 30, rank #1428 of 1776 rated models, from 24 benchmark results.

Benchmark	Score	Metric	Percentile
Open LLM Leaderboard - MuSR	14.13	Score	77.3
Open LLM Leaderboard - IFEval	56.68	Score	68.5
Open Medical LLM - PubMedQA	74.6	Accuracy (%)	62.5
CanAiCode	98.9	Junior-v2 Python Pass Rate (%)	60.2
Open LLM Leaderboard - BBH	29.43	Score	49.7
Open Medical LLM - MMLU College Medicine	61.85	Accuracy (%)	48
Open CoT - LSAT Logical Reasoning	9.41	CoT Gain (%)	45
Open Medical LLM - MMLU Professional Medicine	68.01	Accuracy (%)	43.2
Open CoT - LSAT Reading Comprehension	10.41	CoT Gain (%)	42.7
Open CoT - LSAT Analytical Reasoning	3.04	CoT Gain (%)	41.2
Open Medical LLM - MMLU College Biology	70.83	Accuracy (%)	37.2
Open Medical LLM - MMLU Clinical Knowledge	67.55	Accuracy (%)	36.9