gpt-sw3-40B: benchmark results
AI Sweden's gpt-sw3-40b, a decoder-only causal language model developed with RISE and WASP WARA for Media and Language using NeMo Megatron GPT. Its card describes Nordic-language, English and programming-code pretraining. Provider: AI Sweden. Released 2023-02-22. Access: Open.
Aggregate Rating 1226 ± 29, rank #1567 of 1656 rated models, from 50 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EuroEval Swedish NLU - Multi Wiki QA SV | 71.21 | Character F1 (%) | 64.4 |
| EuroEval Danish NLU - Multi Wiki QA DA | 73.22 | Character F1 (%) | 61 |
| EuroEval Norwegian Knowledge - Idioms NO | 0.89 | MCC (x100) | 34.9 |
| EuroEval Icelandic Common Sense Reasoning | 2.08 | Common Sense Reasoning Average Score (%) | 33 |
| Open Portuguese LLM - FaQuAD NLI | 51.25 | Macro F1 (%) | 32.7 |
| EuroEval Swedish NLU - ScaLA SV | 8.21 | Matthews correlation coefficient (%) | 29.8 |
| EuroEval Swedish NLU | 41.57 | NLU Average Score (%) | 29.6 |
| EuroEval Danish NLU | 27.13 | NLU Average Score (%) | 27.9 |
| EuroEval Danish NLU - ScaLA DA | 3.72 | Matthews correlation coefficient (%) | 26.5 |
| EuroEval Danish Knowledge - Danish Citizen Tests | 18.39 | MCC (x100) | 25.2 |
| EuroEval Danish Knowledge | 13.57 | Knowledge Average Score (%) | 24.3 |
| Open Portuguese LLM - BLUEX | 28.09 | Accuracy (%) | 23.9 |
Interactive version: theaggregate.ai/model?slug=gpt-sw3-40b · How It Works · Data refreshed daily, snapshot 2026-10-20.