mistral-orpo-capybara-7k: benchmark results
KAIST's Mistral-ORPO-Capybara-7k, an ORPO fine-tune of Mistral 7B v0.1 (skipping SFT warmup) trained on 7,000 distilled Capybara multi-turn conversation pairs. Provider: KAIST. Released 2024-03-23. Access: Open.
Aggregate Rating 1380 ± 15, rank #1294 of 1656 rated models, from 35 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-IFEval | 36.02 | Strict accuracy, prompt/instruction mean (%) | 74.7 |
| Open LLM Leaderboard v1 - HellaSwag | 85.34 | Normalized accuracy (%) (10-shot) | 73.7 |
| Open Korean LLM - KorNAT-Helpful | 48.6 | Normalized accuracy (%) | 70.2 |
| Open LLM Leaderboard v1 - WinoGrande | 79.56 | Accuracy (%) (5-shot) | 69.8 |
| Open LLM Leaderboard - IFEval | 53.67 | Score | 64.2 |
| Open LLM Leaderboard v1 - ARC Challenge | 63.48 | Normalized accuracy (%) (25-shot) | 62.9 |
| Open Korean LLM - Ko-GPQA-Diamond | 24.24 | Normalized accuracy (%) | 62.4 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 27.52 | Exact match, flexible extract (%) | 61.7 |
| Open LLM Leaderboard v1 - MMLU | 63.41 | Accuracy (%) (5-shot) | 60.5 |
| Open Portuguese LLM - OAB Exams | 44.37 | Accuracy (%) | 58.1 |
| Open LLM Leaderboard v1 - GSM8K | 42.38 | Accuracy (%) (5-shot) | 57 |
| Open Portuguese LLM - BLUEX | 53.41 | Accuracy (%) | 55.9 |
Interactive version: theaggregate.ai/model?slug=mistral-orpo-capybara-7k · How It Works · Data refreshed daily, snapshot 2026-10-20.