Devstral Small (2512): benchmark results
Provider: Mistral. Released 2025-12-09.
Aggregate Rating 1652 ± 39, rank #610 of 10916 rated models, from 13 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| TRAP - Text Privacy | 18 | Privacy score (%, 1 minus leakage) on the 100 text-document | 90.5 |
| TRAP - Image Task Accuracy | 71.5 | Task accuracy (%) on the 200 image-document scenarios of 500 | 88.1 |
| TRAP - Task Accuracy | 73 | Task accuracy (%): correct tool and arguments using the priv | 85.7 |
| TRAP - Multimodal Task Accuracy | 70.5 | Task accuracy (%) on the 200 multimodal-document scenarios o | 81 |
| TRAP | 25.2 | Overall HM (self-reported) | 68.8 |
| TRAP - Multimodal Privacy | 14.5 | Privacy score (%, 1 minus leakage) on the 200 multimodal-doc | 57.1 |
| TRAP - Privacy | 15.2 | Privacy score (%): 1 minus the leakage rate under the attack | 57.1 |
| SWE-bench Verified (SWE-ABS Tests) | 40.8 | Resolve rate (%) on the 500 SWE-bench Verified tasks when ea | 42.6 |
| SWE-bench Verified | 56.4 | Resolved (%) | 41.3 |
| TRAP - Text Task Accuracy | 81 | Task accuracy (%) on the 100 text-document scenarios of 500 | 40.5 |
| Wolfram LLM Benchmarking Project | 33.3 | Correct Functionality (%) | 33 |
| TRAP - Image Privacy | 14.5 | Privacy score (%, 1 minus leakage) on the 200 image-document | 14.3 |
Interactive version: theaggregate.ai/model?slug=devstral-small-2512 · How It Works · Data refreshed daily, snapshot 2026-11-08.