MagpieLM-8B-SFT-v0.1: benchmark results
Magpie's MagpieLM-8B-SFT-v0.1, an SFT-only fine-tune of Llama 3.1 8B on Magpie-generated instruction data, the intermediate checkpoint before the DPO-aligned MagpieLM-8B-Chat-v0.1. Provider: Magpie. Released 2024-09-15. Access: Open.
Aggregate Rating 1395 ± 21, rank #1235 of 1654 rated models, from 20 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - KorNAT-Helpful | 51.6 | Normalized accuracy (%) | 91.3 |
| Open Korean LLM - KorNAT-Harmless | 65.96 | Normalized accuracy (%) | 83.7 |
| Open Korean LLM - Ko-Winogrande | 63.22 | Accuracy (%) | 73 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 37.91 | Exact match, flexible extract (%) | 72.3 |
| Open Korean LLM - Ko-IFEval | 34.96 | Strict accuracy, prompt/instruction mean (%) | 71.8 |
| Open Korean LLM Leaderboard | 38.78 | Average Score (%) | 68.2 |
| Open LLM Leaderboard - IFEval | 47.21 | Score | 54.3 |
| Open Korean LLM - Ko-EQ-Bench | 0.68 | EQ-Bench score | 46.2 |
| Open Portuguese LLM - FaQuAD NLI | 60.34 | Macro F1 (%) | 44.4 |
| Open Portuguese LLM - ENEM | 57.87 | Accuracy (%) | 42.4 |
| Open Portuguese LLM - BLUEX | 48.4 | Accuracy (%) | 42.2 |
| Open Korean LLM - Ko-GPQA-Diamond | 22.73 | Normalized accuracy (%) | 40.9 |
Interactive version: theaggregate.ai/model?slug=magpielm-8b-sft-v0-1 · How It Works · Data refreshed daily, snapshot 2026-10-24.