Atlas-Chat-2B: benchmark results
Provider: Other. Released 2024-09-26. Access: Open.
Aggregate Rating 1381 ± 33, rank #1284 of 1656 rated models, from 36 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| QIMMA - ArabCulture | 48.72 | Accuracy (%, log-likelihood multiple choice) | 59.8 |
| QIMMA - MizanQA | 54.87 | Normalized multiple-choice probability score (%) | 57.3 |
| Granite Arabic Card - Question Answering - Alyah (Dial) | 62.92 | Exact Match (%) | 55.6 |
| Granite Arabic Card - Question Answering - PalmX 2025 (MSA) | 61.07 | Exact Match (%) | 55.6 |
| Granite Arabic Card - Translation - Dial.→En | 27.71 | BLEU (0–100) | 55.6 |
| Granite Arabic Card - Translation - MSA→En | 32.38 | BLEU (0–100) | 44.4 |
| QIMMA - FannOrFlop | 24.07 | F1 (%) | 42.7 |
| QIMMA - PALMX | 62.29 | Accuracy (%, log-likelihood multiple choice) | 42.7 |
| QIMMA - MedAraBench | 32.46 | Accuracy (%, log-likelihood multiple choice) | 39 |
| QIMMA - MedArabiQ MCQ | 30.8 | Accuracy (%, log-likelihood multiple choice) | 34.1 |
| Granite Arabic Card - Question Answering - Belebele (Dial) | 56.29 | Exact Match (%) | 33.3 |
| QIMMA - 3LM STEM | 62.71 | Accuracy (%, log-likelihood multiple choice) | 32.9 |
Interactive version: theaggregate.ai/model?slug=atlas-chat-2b · How It Works · Data refreshed daily, snapshot 2026-10-20.