gpt-sw3-356m: benchmark results
AI Sweden's gpt-sw3-356m, a decoder-only causal language model developed with RISE and WASP WARA for Media and Language using NeMo Megatron GPT. Its card describes Nordic-language, English and programming-code pretraining. Provider: AI Sweden. Access: Open.
Aggregate Rating 1167 ± 34, rank #9787 of 10916 rated models, from 97 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EuroEval Icelandic NLU - ScaLA IS | 1.57 | Matthews correlation coefficient (%) | 40.3 |
| EuroEval Icelandic Common Sense Reasoning | 2.56 | Common Sense Reasoning Average Score (%) | 34.1 |
| EuroEval Icelandic NLU - NQII | 32.08 | Character F1 (%) | 32.2 |
| EuroEval Swedish NLU - Multi Wiki QA SV | 36.94 | Character F1 (%) | 28.9 |
| EuroEval Norwegian NLU - Norquad | 34.55 | Character F1 (%) | 28 |
| EuroEval Danish NLU - Multi Wiki QA DA | 35.67 | Character F1 (%) | 27.9 |
| EuroEval English NLU - SST-5 | 53.65 | Matthews correlation coefficient (%) | 22.5 |
| EuroEval English NLU - SQuAD | 51.18 | Character F1 (%) | 20.9 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 42.55 | MC2 (%) (0-shot) | 20.9 |
| EuroEval Faroese NLU - FoQA | 9.43 | Character F1 (%) | 19.9 |
| EuroEval Norwegian | 11.77 | Average Score (%) | 19.7 |
| EuroEval Icelandic NLU - MIM-GOLD NER | 18.47 | Micro F1 excluding MISC (%) | 19.4 |
Interactive version: theaggregate.ai/model?slug=gpt-sw3-356m · How It Works · Data refreshed daily, snapshot 2026-11-08.