Mistral 7B Instruct (v0.3): benchmark results
Mistral 7B Instruct v0.3 checkpoint. Provider: Mistral. Released 2024-05-22. Access: Open.
Aggregate Rating 1385 ± 6, rank #1274 of 1656 rated models, from 1291 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| AbstentionBench - underspecified context - MediQ - F1 | 57.82 | Abstention F1 (%) | 100 |
| AbstentionBench - underspecified context - MediQ - Recall | 41.45 | Abstention Recall (%) | 100 |
| ESG-Bench | 80.67 | Overall accuracy (%) on ESG-Bench's 300 test questions over | 100 |
| EpiQAL - Masked-Input Reasoning | 80.8 | Exact Match (%; zero-shot, no CoT, full article) | 100 |
| EpiQAL - Multi-Step Inference | 78.9 | Exact Match (%; zero-shot, no CoT, full article) | 100 |
| HELM AIR-Bench 2024 - #4.3: Adversely affects legal rights | 93.33 | Refusal Rate (%) | 100 |
| HELM TORR - Tablebench Numerical Reasoning - Robustness | 71.09 | Robustness (0-100) | 100 |
| LA Leaderboard - AQuAS | 73.27 | Semantic Answer Similarity (x100) | 100 |
| LA Leaderboard - PAWS Catalan | 72.45 | Accuracy (%) | 100 |
| MultiCaption (Zero-Shot) | 56 | Accuracy (%; balanced test split, 4,920 claim pairs in 52 la | 100 |
| MultiCaption (Zero-Shot) - English Translations | 64.4 | Accuracy (%; the same 4,920 test pairs with the claims in th | 100 |
| PEC-Home (Zero-Shot) - Dynamic Preferences Level 1 | 57.16 | Execution accuracy (%; 712 smart-home commands at ellipsis l | 100 |
Interactive version: theaggregate.ai/model?slug=mistral-7b-instruct-v0-3 · How It Works · Data refreshed daily, snapshot 2026-10-20.