Mistral Small 3: benchmark results
Mistral AI's Small 3, a 24B dense model (January 2025). Provider: Mistral. Released 2025-01-30. Access: Open.
Aggregate Rating 1545 ± 8, rank #627 of 1656 rated models, from 485 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Grip on LLMs - Dutch tinyTruthfulQA | 78 | Accuracy (%; GP-IRT estimate from 100 tinyBenchmarks items, | 100 |
| Grip on LLMs - HonestCityBench - Outdated Information | 66 | Appropriate acknowledgement rate (%; LLM judge) | 100 |
| OmniToM - Belief Extraction - Ambiguous Story Task | 52.97 | Belief-extraction macro F1 (%): story-level F1 between the e | 100 |
| OmniToM - Belief Extraction - Faux-pas Recognition Test | 59.79 | Belief-extraction macro F1 (%): story-level F1 between the e | 100 |
| OmniToM - Belief Labeling - Knowledge Access | 74.59 | Belief-labeling accuracy (%): exact-match accuracy of the pr | 100 |
| Open PL LLM - PoQuAD Open Book (generative, 5-shot) | 73.99 | Levenshtein Similarity (%) | 99.7 |
| EuroEval Italian Common Sense Reasoning | 82.73 | Common Sense Reasoning Average Score (%) | 98.9 |
| EuroEval French Common Sense Reasoning | 83.43 | Common Sense Reasoning Average Score (%) | 98.8 |
| EuroEval German NLU - Sb10K | 62.27 | Matthews correlation coefficient (%) | 98.8 |
| Open PL LLM - PolEmo2-IN (generative, 5-shot) | 87.67 | Accuracy (%) | 97.5 |
| MT-Bench PL - Extraction | 9.9 | Judge Score (0-10) | 96.9 |
| Open PL LLM - Belebele (generative, 5-shot) | 92 | Accuracy (%) | 96.9 |
Interactive version: theaggregate.ai/model?slug=mistral-small-3 · How It Works · Data refreshed daily, snapshot 2026-10-20.