Mistral_Pro_8B_v0.1: benchmark results
Tencent ARC Lab's Mistral-Pro-8B, an 8B expansion of Mistral 7B with added Transformer blocks, further trained on code and math corpora. Provider: Tencent. Released 2024-02-22. Access: Open.
Aggregate Rating 1371 ± 29, rank #1298 of 1635 rated models, from 17 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard - MuSR | 42.42 | Score | 64.7 |
| Open LLM Leaderboard v1 - ARC Challenge | 62.2 | Normalized accuracy (%) (25-shot) | 56.6 |
| Open LLM Leaderboard v1 - MMLU | 61.74 | Accuracy (%) (5-shot) | 52.7 |
| Open LLM Leaderboard v1 - WinoGrande | 76.8 | Accuracy (%) (5-shot) | 50.7 |
| Open LLM Leaderboard v1 - GSM8K | 34.19 | Accuracy (%) (5-shot) | 50 |
| Open Portuguese LLM - ENEM | 61.37 | Accuracy (%) | 49.3 |
| Open LLM Leaderboard v1 - HellaSwag | 82.13 | Normalized accuracy (%) (10-shot) | 49 |
| Open Portuguese LLM - OAB Exams | 40.77 | Accuracy (%) | 45.2 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 49.32 | MC2 (%) (0-shot) | 44.4 |
| Open Portuguese LLM - BLUEX | 48.82 | Accuracy (%) | 43.5 |
| Open Portuguese LLM - ASSIN2 RTE | 85.74 | Macro F1 (%) | 37 |
| Open LLM Leaderboard - GPQA | 28.02 | Score | 35.2 |
Interactive version: theaggregate.ai/model?slug=mistral-pro-8b-v0-1 · How It Works · Data refreshed daily, snapshot 2026-11-05.