Devstral (2512): benchmark results
Provider: Mistral. Released 2025-12-09.
Aggregate Rating 1638 ± 28, rank #699 of 10916 rated models, from 7 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Wolfram LLM Benchmarking Project | 48 | Correct Functionality (%) | 58.4 |
| SWE-bench Verified (SWE-ABS Tests) | 40.6 | Resolve rate (%) on the 500 SWE-bench Verified tasks when ea | 37 |
| SWE-bench Verified | 53.8 | Resolved (%) | 30.4 |
| Boiling the Frog | 50.6 | Strict ASR (self-reported) | 25 |
| Boiling the Frog - Safe Agency Score | 10.7 | Safe Agency Score (SAS, %): benign strict success rate times | 25 |
| ObviousBench | 65.97 | Answer pass³ (%) | 22.3 |
| IOI | 0.67 | Score (imported) | 2.5 |
Interactive version: theaggregate.ai/model?slug=devstral-2512 · How It Works · Data refreshed daily, snapshot 2026-11-08.