gpt-sw3-356m-instruct: benchmark results
AI Sweden's gpt-sw3-356m-instruct, a decoder-only causal language model developed with RISE and WASP WARA for Media and Language using NeMo Megatron GPT. Its card describes Nordic-language, English and programming-code pretraining. This instruction-tuned release uses instruction data in chat and raw-text formats and remains distinct from the corresponding base checkpoint. Provider: AI Sweden. Released 2023-04-28. Access: Open.
Aggregate Rating 1161 ± 30, rank #9828 of 10916 rated models, from 97 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EuroEval Icelandic NLU - NQII | 33.29 | Character F1 (%) | 34.7 |
| EuroEval Icelandic Common Sense Reasoning | 1.39 | Common Sense Reasoning Average Score (%) | 26.9 |
| EuroEval Norwegian NLU - Norquad | 22.37 | Character F1 (%) | 20.4 |
| EuroEval Faroese NLU - FONE | 36.75 | Micro F1 excluding MISC (%) | 19.5 |
| EuroEval Faroese NLU - FoQA | 6.96 | Character F1 (%) | 18.4 |
| Open LLM Leaderboard v1 - GSM8K | 1.74 | Accuracy (%) (5-shot) | 18.2 |
| EuroEval Icelandic NLU | 13.61 | NLU Average Score (%) | 18.1 |
| EuroEval Swedish NLU - Multi Wiki QA SV | 17.03 | Character F1 (%) | 17.8 |
| EuroEval Icelandic NLU - ScaLA IS | 0 | Matthews correlation coefficient (%) | 17.4 |
| EuroEval Icelandic NLU - MIM-GOLD NER | 16.61 | Micro F1 excluding MISC (%) | 16.9 |
| EuroEval Icelandic NLU - Hotter and Colder Sentiment | 4.54 | Matthews correlation coefficient (%) | 16.8 |
| EuroEval Norwegian | 8.14 | Average Score (%) | 16 |
Interactive version: theaggregate.ai/model?slug=gpt-sw3-356m-instruct · How It Works · Data refreshed daily, snapshot 2026-11-08.