Synatra-7B-v0.3-RP: benchmark results
maywell's Synatra 7B v0.3-RP, a roleplay-oriented fine-tune of Mistral 7B Instruct v0.1 from his personal project, non-commercial license. Provider: maywell. Released 2023-10-29. Access: Open.
Aggregate Rating 1379 ± 16, rank #1301 of 1656 rated models, from 29 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-GPQA-Diamond | 25.25 | Normalized accuracy (%) | 75.9 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 34.8 | Exact match, flexible extract (%) | 68 |
| Open Korean LLM Leaderboard | 38.43 | Average Score (%) | 67 |
| Open Ko-LLM Leaderboard | 38.43 | Average (%) | 66.9 |
| Open Korean LLM - Ko-EQ-Bench | 17.86 | EQ-Bench score | 64 |
| Open Korean LLM - KorNAT-CKA | 25.72 | Normalized accuracy (%) | 60.1 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 52.64 | MC2 (%) (0-shot) | 56.8 |
| Open LLM Leaderboard v1 - ARC Challenge | 62.2 | Normalized accuracy (%) (25-shot) | 56.6 |
| Open Portuguese LLM - ASSIN2 RTE | 89.82 | Macro F1 (%) | 52.4 |
| Open LLM Leaderboard v1 - HellaSwag | 82.29 | Normalized accuracy (%) (10-shot) | 50.1 |
| Open LLM Leaderboard v1 - MMLU | 60.8 | Accuracy (%) (5-shot) | 49.8 |
| Open LLM Leaderboard v1 - WinoGrande | 76.48 | Accuracy (%) (5-shot) | 48.5 |
Interactive version: theaggregate.ai/model?slug=synatra-7b-v0-3-rp · How It Works · Data refreshed daily, snapshot 2026-10-20.