Yarn-Mistral-7B-128k: benchmark results
YaRN project's continued-pretrained Mistral 7B release with a 32,768-token extended context window. It remains distinct from the original base checkpoint and other YaRN context variants. Provider: Nous Research. Released 2023-10-31. Access: Open.
Aggregate Rating 1342 ± 21, rank #1403 of 1661 rated models, from 24 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Hallucinations Leaderboard | 46.29 | Average Task Score (score) | 68.4 |
| Open LLM Leaderboard v1 - MMLU | 63.02 | Accuracy (%) (5-shot) | 57.9 |
| Open LLM Leaderboard - GPQA | 29.87 | Score | 54.6 |
| Open LLM Leaderboard v1 - WinoGrande | 76.95 | Accuracy (%) (5-shot) | 52 |
| Open LLM Leaderboard v1 - HellaSwag | 82.5 | Normalized accuracy (%) (10-shot) | 51.7 |
| Open LLM Leaderboard v1 - GSM8K | 32.6 | Accuracy (%) (5-shot) | 48.9 |
| Open LLM Leaderboard - MuSR | 40.71 | Score | 47.8 |
| Open LLM Leaderboard v1 - ARC Challenge | 59.64 | Normalized accuracy (%) (25-shot) | 44.3 |
| BABILong | 32.11 | Mean <=128k (%) | 37.9 |
| Open LLM Leaderboard - MMLU-Pro | 28.93 | Score | 33.2 |
| Open LLM Leaderboard - BBH | 43.14 | Score | 29.8 |
| BABILong (NIAH, v0, 32k) | 16 | QA1–5 Mean Accuracy (%) | 22.9 |
Interactive version: theaggregate.ai/model?slug=yarn-mistral-7b-128k · How It Works · Data refreshed daily, snapshot 2026-10-23.