Calme-4x7B-MoE-v0.2: benchmark results
Second iteration of MaziyarPanahi's Calme 4x7B MoE of his Calme-7B Mistral fine-tunes, with higher ARC and GSM8K scores than v0.1. Provider: Other. Released 2024-03-17. Access: Open.
Unified ELO 1474 ± 1, rank #820 of 1392 rated models, from 15 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Chinese LLM - TruthfulQA MC | 61.02 | Accuracy (%) | 85.8 |
| Open Korean LLM Leaderboard | 43.12 | Average Score (%) | 85.6 |
| Open LLM Leaderboard - MuSR | 12.54 | Score | 68.7 |
| Open Chinese LLM - GSM8K | 50.19 | Accuracy (%) | 68.2 |
| Open LLM Leaderboard - BBH | 31.4 | Score | 56.9 |
| Open Chinese LLM - WinoGrande | 62.83 | Accuracy (%) | 49.1 |
| Open Chinese LLM - ARC Challenge | 52.05 | Accuracy (%) | 48.1 |
| Open Chinese LLM Leaderboard | 56.46 | Average Score (%) | 47.6 |
| Open Chinese LLM - HellaSwag | 59.65 | Accuracy (%) | 47.3 |
| Open LLM Leaderboard - IFEval | 42.94 | Score | 44.9 |
| Open LLM Leaderboard - MATH Level 5 | 7.4 | Score | 38.9 |
| Open LLM Leaderboard - MMLU-Pro | 22.86 | Score | 38.5 |
Interactive version: theaggregate.ai/model?slug=calme-4x7b-moe-v0-2 · How It Works · Data refreshed daily, snapshot 2026-09-05.