gpt-sw3-126m-instruct: benchmark results
AI Sweden's gpt-sw3-126m-instruct, a decoder-only causal language model developed with RISE and WASP WARA for Media and Language using NeMo Megatron GPT. Its card describes Nordic-language, English and programming-code pretraining. This instruction-tuned release uses instruction data in chat and raw-text formats and remains distinct from the corresponding base checkpoint. Provider: AI Sweden. Released 2023-04-28. Access: Open.
Aggregate Rating 1153 ± 34, rank #9930 of 10916 rated models, from 97 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EuroEval Icelandic NLU - ScaLA IS | 2.31 | Matthews correlation coefficient (%) | 45.8 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 42.65 | MC2 (%) (0-shot) | 21.2 |
| EuroEval Icelandic Common Sense Reasoning | 0.59 | Common Sense Reasoning Average Score (%) | 19.8 |
| EuroEval Icelandic NLU - MIM-GOLD NER | 17.85 | Micro F1 excluding MISC (%) | 18.7 |
| EuroEval Danish NLU - Angry Tweets | 17.37 | Matthews correlation coefficient (%) | 16.3 |
| EuroEval Faroese NLU - ScaLA FO | 0 | Matthews correlation coefficient (%) | 15.9 |
| EuroEval Norwegian | 8.13 | Average Score (%) | 15.8 |
| Open LLM Leaderboard v1 - GSM8K | 0.99 | Accuracy (%) (5-shot) | 15 |
| EuroEval Dutch Common Sense Reasoning | 0.69 | Common Sense Reasoning Average Score (%) | 14.8 |
| EuroEval Faroese NLU - FONE | 29.27 | Micro F1 excluding MISC (%) | 14.7 |
| EuroEval Norwegian NLU - Norquad | 9.58 | Character F1 (%) | 14.4 |
| EuroEval Icelandic NLU - Hotter and Colder Sentiment | 3.39 | Matthews correlation coefficient (%) | 14 |
Interactive version: theaggregate.ai/model?slug=gpt-sw3-126m-instruct · How It Works · Data refreshed daily, snapshot 2026-11-08.