zephyr-7B-gemma-v0.1: benchmark results
Hugging Face H4's Zephyr 7B Gemma, the third Zephyr release, a DPO fine-tune of Google's Gemma 7B on public and synthetic dialogue data via the Alignment Handbook recipe. Provider: HuggingFace. Released 2024-03-01. Access: Open.
Aggregate Rating 1367 ± 42, rank #1345 of 1656 rated models, from 17 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Portuguese LLM - FaQuAD NLI | 74.86 | Macro F1 (%) | 71.6 |
| RewardBench Chat | 95.81 | Accuracy (%) | 67.6 |
| Open LLM Leaderboard - GPQA | 29.45 | Score | 49.7 |
| Open Portuguese LLM - OAB Exams | 40.73 | Accuracy (%) | 45 |
| Open Portuguese LLM - ENEM | 58.15 | Accuracy (%) | 43.3 |
| Open LLM Leaderboard - MATH Level 5 | 8.16 | Score | 41.4 |
| Open Portuguese LLM - BLUEX | 47.43 | Accuracy (%) | 39.2 |
| Open Portuguese LLM - ASSIN2 RTE | 86.05 | Macro F1 (%) | 37.4 |
| Open LLM Leaderboard - BBH | 46.24 | Score | 36.8 |
| RewardBench Reasoning | 74.63 | Accuracy (%) | 33.7 |
| Open LLM Leaderboard - MMLU-Pro | 28.47 | Score | 32.1 |
| Open LLM Leaderboard - IFEval | 33.64 | Score | 31.6 |
Interactive version: theaggregate.ai/model?slug=zephyr-7b-gemma-v0-1 · How It Works · Data refreshed daily, snapshot 2026-10-20.