SauerkrautLM-Nemo-12B-Instruct: benchmark results
VAGOsolutions' German-English Spectrum fine-tune of Mistral-Nemo-Instruct-2407 (12B), targeting 25% of layers to preserve base knowledge (July 2024). Provider: VAGOsolutions. Released 2024-07-22. Access: Open.
Aggregate Rating 1497 ± 31, rank #796 of 1656 rated models, from 134 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Portuguese LLM - FaQuAD NLI | 83.18 | Macro F1 (%) | 96.2 |
| Open Arabic LLM - Alghafa Multiple Choice Rating Sentiment NO Neutral Task | 81.33 | Accuracy (%) | 90.1 |
| Open Arabic LLM - Arabic MMLU Arabic Language (Grammar) | 53.15 | Accuracy (%) | 84 |
| Open Portuguese LLM - OAB Exams | 52.53 | Accuracy (%) | 81.9 |
| Open LLM Leaderboard - MuSR | 44.69 | Score | 81.8 |
| Open Portuguese LLM - ASSIN2 RTE | 92.65 | Macro F1 (%) | 81.7 |
| Open Arabic LLM - Alghafa Multiple Choice Rating Sentiment Task | 54.51 | Accuracy (%) | 81.2 |
| Open Arabic LLM - Madinah QA Arabic Language (Grammar) | 54.79 | Accuracy (%) | 80.6 |
| Open Arabic LLM - Alghafa Multiple Choice Sentiment Task | 41.22 | Accuracy (%) | 77.8 |
| Open Portuguese LLM - ENEM | 70.05 | Accuracy (%) | 74.9 |
| Open LLM Leaderboard - IFEval | 61.13 | Score | 73.2 |
| Open Portuguese LLM - BLUEX | 58.41 | Accuracy (%) | 73.2 |
Interactive version: theaggregate.ai/model?slug=sauerkrautlm-nemo-12b-instruct · How It Works · Data refreshed daily, snapshot 2026-10-20.