Mistral Nemo Instruct (2407): benchmark results
July 2024 Mistral Nemo Instruct checkpoint. Provider: Mistral. Released 2024-07-18. Access: Open.
Aggregate Rating 1460 ± 8, rank #936 of 1656 rated models, from 378 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| BTZSC - Topic | 69.05 | Macro-F1 (%) | 100 |
| HumanLikeness - Meaning-2 | 75.1 | Humanlike Score (%) | 100 |
| BTZSC | 66.97 | Macro-F1 (%) | 97.1 |
| OpenCompass Language - Creation - English (CompassBench 2409) | 55.9 | Score (%) | 96.6 |
| CARTE - Corse | 85 | Accuracy (%) on the 100 CARTE questions about Corse (Corsica | 96.2 |
| CARTE - Ile-de-France | 81 | Accuracy (%) on the 273 CARTE questions about Ile-de-France, | 96.2 |
| CARTE - Nouvelle-Aquitaine | 67.8 | Accuracy (%) on the 180 CARTE questions about Nouvelle-Aquit | 96.2 |
| CARTE-LV | 74.2 | Accuracy (%) on the 233 CARTE-LV questions on regional lingu | 96.2 |
| Open Portuguese LLM - FaQuAD NLI | 82.34 | Macro F1 (%) | 94.3 |
| CARTE - Pays de la Loire | 69.2 | Accuracy (%) on the 133 CARTE questions about Pays de la Loi | 94.2 |
| CARTE - Provence-Alpes-Cote d'Azur | 69.6 | Accuracy (%) on the 224 CARTE questions about Provence-Alpes | 94.2 |
| CARTE | 73.6 | Accuracy (%) on all 2,431 CARTE questions (question-weighted | 92.3 |
Interactive version: theaggregate.ai/model?slug=mistral-nemo-instruct-2407 · How It Works · Data refreshed daily, snapshot 2026-10-20.