Llama 3.1 8B CPT SEA LIONv3 Instruct: benchmark results
AI Singapore's Llama SEA-LION v3 8B IT, a Llama 3.1 8B continually pretrained and instruction-tuned for Southeast Asian languages (Burmese, Indonesian, Khmer, Thai, Vietnamese and more), with 128K context. Provider: AI Singapore. Released 2024-12-11. Access: Open.
Aggregate Rating 1491 ± 25, rank #810 of 1656 rated models, from 73 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| HELM SeaHELM - XQuAD (Thai) | 73.65 | SQuAD macro-averaged F1 score | 95 |
| HELM SeaHELM - XQuAD (Vietnamese) | 58.37 | SQuAD macro-averaged F1 score | 90 |
| Thai LLM - Writing | 7.45 | Rating (0-10) | 87.3 |
| Thai LLM NLG - iapp_squad_seacrowd_qa (ROUGE1) | 63.43 | ROUGE1 | 85.1 |
| Thai LLM NLG - iapp_squad_seacrowd_qa (ROUGEL) | 62.8 | ROUGEL | 85.1 |
| Thai LLM NLG - iapp_squad_seacrowd_qa (ROUGE2) | 50.73 | ROUGE2 | 83.6 |
| HELM SeaHELM - Flores (th-en) | 57.09 | ChrF++ | 80 |
| Thai LLM NLU - wisesight_thai_sentiment_seacrowd_text | 49.79 | Accuracy (%) | 78.3 |
| Thai LLM - Roleplay | 7.5 | Rating (0-10) | 78.2 |
| Thai LLM - Coding | 7.05 | Rating (0-10) | 76.1 |
| HELM SeaHELM - Flores (id-en) | 65.22 | ChrF++ | 75 |
| Thai LLM NLG - flores200_tha_Thai_eng_Latn_seacrowd_t2t (SacreBLEU) | 33.3 | SacreBLEU | 71.6 |
Interactive version: theaggregate.ai/model?slug=llama-3-1-8b-cpt-sea-lionv3-instruct · How It Works · Data refreshed daily, snapshot 2026-10-20.