calme-2.4-qwen2-7B: benchmark results
MaziyarPanahi's calme-2.4, a fine-tune of Qwen2 7B using the ChatML prompt template. Provider: MaziyarPanahi. Released 2024-06-27. Access: Open.
Aggregate Rating 1416 ± 19, rank #1168 of 1661 rated models, from 21 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-GSM8K (flexible extract) | 53.98 | Exact match, flexible extract (%) | 87.9 |
| Open Portuguese LLM - OAB Exams | 53.21 | Accuracy (%) | 83.1 |
| Open LLM Leaderboard - MuSR | 44.53 | Score | 81.1 |
| Open Portuguese LLM - BLUEX | 61.75 | Accuracy (%) | 81 |
| Open Portuguese LLM - ASSIN2 RTE | 92.47 | Macro F1 (%) | 79.2 |
| Open Portuguese LLM - ENEM | 70.82 | Accuracy (%) | 77.7 |
| Open Korean LLM Leaderboard | 40.37 | Average Score (%) | 76.6 |
| Open Ko-LLM Leaderboard | 40.37 | Average (%) | 76.5 |
| Open LLM Leaderboard - MMLU-Pro | 39.77 | Score | 73.3 |
| Open LLM Leaderboard - MATH Level 5 | 20.32 | Score | 71.5 |
| Open Korean LLM - Ko-EQ-Bench | 16.87 | EQ-Bench score | 62.3 |
| Open Portuguese LLM - FaQuAD NLI | 69.94 | Macro F1 (%) | 60 |
Interactive version: theaggregate.ai/model?slug=calme-2-4-qwen2-7b · How It Works · Data refreshed daily, snapshot 2026-10-23.