Synatra-7B-v0.3-dpo: benchmark results
maywell's Synatra 7B v0.3-dpo, a personal-project DPO fine-tune of Mistral 7B Instruct v0.1, trained on a single A100. Provider: maywell. Released 2023-11-08. Access: Open.
Aggregate Rating 1409 ± 16, rank #1193 of 1656 rated models, from 25 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-EQ-Bench | 35.39 | EQ-Bench score | 85.4 |
| Open Korean LLM Leaderboard | 41.07 | Average Score (%) | 80.4 |
| Open Ko-LLM Leaderboard | 41.07 | Average (%) | 80.3 |
| Open Korean LLM - KorNAT-Harmless | 64.36 | Normalized accuracy (%) | 73.5 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 37.83 | Exact match, flexible extract (%) | 72.1 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 56.46 | MC2 (%) (0-shot) | 68.1 |
| Open Korean LLM - Ko-Winogrande | 61.88 | Accuracy (%) | 64.9 |
| Open LLM Leaderboard v1 - ARC Challenge | 62.8 | Normalized accuracy (%) (25-shot) | 59.8 |
| Open Korean LLM - KorNAT-Helpful | 47.28 | Normalized accuracy (%) | 55 |
| Open LLM Leaderboard v1 - HellaSwag | 82.58 | Normalized accuracy (%) (10-shot) | 52.5 |
| Open LLM Leaderboard v1 - MMLU | 61.46 | Accuracy (%) (5-shot) | 51.9 |
| Open Korean LLM - KorNAT-CKA | 24.05 | Normalized accuracy (%) | 49.2 |
Interactive version: theaggregate.ai/model?slug=synatra-7b-v0-3-dpo · How It Works · Data refreshed daily, snapshot 2026-10-20.