magnum-v4-9B: benchmark results
Anthracite's community magnum v4 fine-tune of Gemma 2 9B aiming to replicate Claude 3 Sonnet/Opus prose quality. Provider: anthracite-org. Released 2024-10-20. Access: Open.
Aggregate Rating 1512 ± 20, rank #750 of 1661 rated models, from 21 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-EQ-Bench | 56.61 | EQ-Bench score | 97 |
| Open Korean LLM Leaderboard | 48.19 | Average Score (%) | 95.8 |
| Open Ko-LLM Leaderboard | 48.19 | Average (%) | 95.7 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 59.82 | Exact match, flexible extract (%) | 93.6 |
| Open Korean LLM - Ko-IFEval | 50.23 | Strict accuracy, prompt/instruction mean (%) | 93.5 |
| Open LLM Leaderboard - GPQA | 34.73 | Score | 87.1 |
| Open Korean LLM - KorNAT-CKA | 31.01 | Normalized accuracy (%) | 85.8 |
| Open Korean LLM - Ko-GPQA-Diamond | 26.26 | Normalized accuracy (%) | 85.5 |
| Open LLM Leaderboard - MuSR | 45.16 | Score | 84.2 |
| Open LLM Leaderboard - MMLU-Pro | 39.53 | Score | 72.7 |
| Open LLM Leaderboard - BBH | 53.36 | Score | 65.7 |
| Open Korean LLM - Ko-Winogrande | 61.09 | Accuracy (%) | 59.7 |
Interactive version: theaggregate.ai/model?slug=magnum-v4-9b · How It Works · Data refreshed daily, snapshot 2026-10-23.