gpt-sw3-20B-instruct: benchmark results
AI Sweden's gpt-sw3-20b-instruct, a decoder-only causal language model developed with RISE and WASP WARA for Media and Language using NeMo Megatron GPT. Its card describes Nordic-language, English and programming-code pretraining. This instruction-tuned release uses instruction data in chat and raw-text formats and remains distinct from the corresponding base checkpoint. Provider: AI Sweden. Released 2023-04-28. Access: Open.
Aggregate Rating 1261 ± 32, rank #9152 of 10916 rated models, from 55 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EuroEval Icelandic NLU - NQII | 53.88 | Character F1 (%) | 81.5 |
| EuroEval English NLU - SQuAD | 83.08 | Character F1 (%) | 74.2 |
| EuroEval Norwegian NLU - Norquad | 67.93 | Character F1 (%) | 64.4 |
| EuroEval English NLU - SST-5 | 65.58 | Matthews correlation coefficient (%) | 56.9 |
| EuroEval Icelandic | 25.68 | Average Score (%) | 45.7 |
| EuroEval Icelandic NLU - ScaLA IS | 1.95 | Matthews correlation coefficient (%) | 44.1 |
| EuroEval Norwegian NLU - Norec | 40.89 | Matthews correlation coefficient (%) | 43.1 |
| EuroEval Norwegian NLU - ScaLA NN | 8.32 | Matthews correlation coefficient (%) | 37.6 |
| EuroEval Icelandic NLU | 25.68 | NLU Average Score (%) | 35.8 |
| EuroEval Swedish NLU - Swerec | 68.23 | Matthews correlation coefficient (%) | 33.5 |
| EuroEval Swedish NLU - ScaLA SV | 12.39 | Matthews correlation coefficient (%) | 33 |
| EuroEval Norwegian NLU - ScaLA NB | 9.45 | Matthews correlation coefficient (%) | 32.1 |
Interactive version: theaggregate.ai/model?slug=gpt-sw3-20b-instruct · How It Works · Data refreshed daily, snapshot 2026-11-08.