gpt-sw3-1.3B-instruct: benchmark results
AI Sweden's gpt-sw3-1.3b-instruct, a decoder-only causal language model developed with RISE and WASP WARA for Media and Language using NeMo Megatron GPT. Its card describes Nordic-language, English and programming-code pretraining. This instruction-tuned release uses instruction data in chat and raw-text formats and remains distinct from the corresponding base checkpoint. Provider: AI Sweden. Released 2023-04-28. Access: Open.
Aggregate Rating 1183 ± 34, rank #9702 of 10916 rated models, from 20 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EuroEval Swedish NLU - Swerec | 73.34 | Matthews correlation coefficient (%) | 42.5 |
| EuroEval Norwegian NLU - Norec | 35.58 | Matthews correlation coefficient (%) | 36.8 |
| EuroEval Danish NLU - Angry Tweets | 27.14 | Matthews correlation coefficient (%) | 24.6 |
| Open LLM Leaderboard v1 - GSM8K | 1.59 | Accuracy (%) (5-shot) | 17.6 |
| EuroEval Danish | 15.44 | Average Score (%) | 16.8 |
| EuroEval Norwegian | 8.88 | Average Score (%) | 16.6 |
| EuroEval Norwegian Knowledge - NRK Quiz QA | 3.36 | MCC (x100) | 16.2 |
| EuroEval Danish NLU - Multi Wiki QA DA | 17.3 | Character F1 (%) | 15.2 |
| EuroEval Swedish NLU - Multi Wiki QA SV | 15.01 | Character F1 (%) | 14.2 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 40.31 | MC2 (%) (0-shot) | 13.6 |
| EuroEval Norwegian Knowledge | 1.29 | Knowledge Average Score (%) | 13.3 |
| Open LLM Leaderboard v1 - MMLU | 26.17 | Accuracy (%) (5-shot) | 11.6 |
Interactive version: theaggregate.ai/model?slug=gpt-sw3-1-3b-instruct · How It Works · Data refreshed daily, snapshot 2026-11-08.