Calme-4x7B-MoE-v0.2: benchmark results
Second iteration of MaziyarPanahi's Calme 4x7B MoE of his Calme-7B Mistral fine-tunes, with higher ARC and GSM8K scores than v0.1. Provider: MaziyarPanahi. Released 2024-03-17. Access: Open.
Aggregate Rating 1449 ± 15, rank #1005 of 1656 rated models, from 35 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - KorNAT-CKA | 35.45 | Normalized accuracy (%) | 95.5 |
| Open LLM Leaderboard v1 - GSM8K | 71.04 | Accuracy (%) (5-shot) | 95 |
| Open Korean LLM - KorNAT-Harmless | 71.61 | Normalized accuracy (%) | 94.4 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 72.84 | MC2 (%) (0-shot) | 93.9 |
| Open LLM Leaderboard v1 - HellaSwag | 88.63 | Normalized accuracy (%) (10-shot) | 93.2 |
| Open LLM Leaderboard v1 - ARC Challenge | 72.35 | Normalized accuracy (%) (25-shot) | 91.7 |
| Open LLM Leaderboard v1 - WinoGrande | 83.03 | Accuracy (%) (5-shot) | 86.8 |
| Open Chinese LLM - TruthfulQA MC | 61.02 | Accuracy (%) | 85.5 |
| Open Korean LLM Leaderboard | 41.35 | Average Score (%) | 82.1 |
| Open Ko-LLM Leaderboard | 41.35 | Average (%) | 82 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 44.96 | Exact match, flexible extract (%) | 79.9 |
| Open LLM Leaderboard v1 - MMLU | 64.64 | Accuracy (%) (5-shot) | 74 |
Interactive version: theaggregate.ai/model?slug=calme-4x7b-moe-v0-2 · How It Works · Data refreshed daily, snapshot 2026-10-20.