SILMA-9B-Instruct-v1.0: benchmark results
SILMA AI's 9B Arabic instruct model built on Google's Gemma 2, claiming top open-weights Arabic performance at release despite its small size (August 2024). Provider: SILMA AI. Released 2024-08-17. Access: Open.
Aggregate Rating 1464 ± 18, rank #913 of 1656 rated models, from 191 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Arabic LLM - Arabic MMLU General Knowledge (Middle School) | 77.91 | Accuracy (%) | 91.4 |
| Open LLM Leaderboard - MuSR | 46.37 | Score | 89.2 |
| QIMMA - ArabCulture | 71.22 | Accuracy (%, log-likelihood multiple choice) | 87.8 |
| Open Arabic LLM - Arabic MMLU Arabic Language (General) | 64.54 | Accuracy (%) | 85.8 |
| ArabCulture-Dialogue - MCQ (MSA) | 78.3 | Accuracy (%) selecting the one culturally appropriate contin | 81.8 |
| MEDIC Benchmark | 73.25 | MEDIC Public Table Average | 78.9 |
| Open Arabic LLM - Aratrust Privacy | 94.74 | Accuracy (%) | 78.3 |
| Open Arabic LLM - Arabic MMLU Computer Science (Primary School) | 75.79 | Accuracy (%) | 77.8 |
| Open Arabic LLM - Arabic MMLU General Knowledge (Primary School) | 72.22 | Accuracy (%) | 77.8 |
| Open Arabic LLM - Arabic MMLU Accounting (University) | 64.86 | Accuracy (%) | 76.5 |
| Open Arabic LLM - Arabic MMLU Civics (High School) | 48.28 | Accuracy (%) | 76.5 |
| Open Arabic LLM - Madinah QA Arabic Language (General) | 75.65 | Accuracy (%) | 74.4 |
Interactive version: theaggregate.ai/model?slug=silma-9b-instruct-v1-0 · How It Works · Data refreshed daily, snapshot 2026-10-20.