SauerkrautLM-Nemo-12B-Instruct — benchmark results
VAGOsolutions' German-English Spectrum fine-tune of Mistral-Nemo-Instruct-2407 (12B), targeting 25% of layers to preserve base knowledge (July 2024). Provider: VAGOsolutions. Released 2024-07-22. Access: Open.
Unified ELO 1469 ± 11, rank #911 of 1776 rated models, from 129 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Arabic LLM - Alghafa Multiple Choice Rating Sentiment NO Neutral Task | 81.33 | Accuracy (%) | 90.1 |
| Open LLM Leaderboard - MuSR | 17.16 | Score | 87.9 |
| Open Arabic LLM - Arabic MMLU Arabic Language (Grammar) | 53.15 | Accuracy (%) | 84 |
| Open Arabic LLM - Alghafa Multiple Choice Rating Sentiment Task | 54.51 | Accuracy (%) | 81.2 |
| Open Arabic LLM - Madinah QA Arabic Language (Grammar) | 54.79 | Accuracy (%) | 80.6 |
| Open Arabic LLM - Alghafa Multiple Choice Sentiment Task | 41.22 | Accuracy (%) | 77.8 |
| Open LLM Leaderboard - IFEval | 61.13 | Score | 73.2 |
| Open Arabic LLM - Alghafa Meta AR MSA | 84.13 | Accuracy (%) | 71 |
| Open Arabic LLM - Alghafa Multiple Choice Grounded Statement Xglue Mlqa Task | 88 | Accuracy (%) | 69.4 |
| Open Arabic LLM - Arabic MMLU HT Formal Logic | 46.03 | Accuracy (%) | 68.5 |
| Open Arabic LLM - Alghafa Average | 73.79 | Accuracy (%) | 67.9 |
| Open LLM Leaderboard - GPQA | 7.94 | Score | 67 |
Interactive version: theaggregate.ai/model?slug=sauerkrautlm-nemo-12b-instruct · How the rankings work · Data refreshed daily, snapshot 2026-07-22.