Devstral Medium: benchmark results
Mistral's API-only mid-tier agentic coding model (July 2025, built with All Hands AI) scoring 61.6% on SWE-Bench Verified at Medium-3 pricing. Provider: Mistral. Released 2025-07-10. Access: API.
Aggregate Rating 1565 ± 27, rank #564 of 1656 rated models, from 72 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| AA-Omniscience Hallucination Rate | 62.19 | Hallucination Rate (%) | 69.6 |
| AA-Omniscience Index - Software Engineering (SWE) - Java | -27 | Omniscience Index | 64.2 |
| AA Omniscience - Software Engineering (SWE) - Dart | 20 | Accuracy (%) | 59.4 |
| AA Omniscience - Software Engineering (SWE) - Julia | 12 | Accuracy (%) | 58 |
| AA-Omniscience Index - Software Engineering (SWE) - Dart | -24 | Omniscience Index | 57.1 |
| AA Omniscience - Software Engineering (SWE) - Java | 16 | Accuracy (%) | 57 |
| AA-Omniscience Index - Software Engineering (SWE) - PHP | -18 | Omniscience Index | 56.9 |
| AA-Omniscience Index - Software Engineering (SWE) - Go | -24 | Omniscience Index | 55.9 |
| AA Omniscience - Software Engineering (SWE) - Kotlin | 18 | Accuracy (%) | 55.3 |
| AA Omniscience - Software Engineering (SWE) - TypeScript | 18.89 | Accuracy (%) | 53.8 |
| AA-Omniscience Index - Software Engineering (SWE) - R | -38 | Omniscience Index | 53.8 |
| AA-Omniscience Index - Software Engineering (SWE) - Python | -26.5 | Omniscience Index | 53.5 |
Interactive version: theaggregate.ai/model?slug=devstral-medium · How It Works · Data refreshed daily, snapshot 2026-10-20.