BioMistral-7B: benchmark results
Avignon University's BioMistral-7B, a biomedical-domain LLM built on Mistral 7B and further pretrained on PubMed Central, evaluated across 10 English medical question-answering tasks and 7 additional languages. Provider: Avignon Univ. Released 2024-02-01. Access: Open.
Aggregate Rating 1316 ± 22, rank #8417 of 10916 rated models, from 71 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| IndoNutriQA - 0-shot EM | 2.92 | EM (%) | 92.9 |
| MHGraphBench - Two-hop Verification | 53.92 | Accuracy (%) on the 1,200 two-hop verification items (about | 78.6 |
| IndoNutriQA - 5-shot EM | 8.03 | EM (%) | 71.4 |
| IndoNutriQA - 0-shot BLEU | 12.07 | BLEU (%) | 57.1 |
| IndoNutriQA - 0-shot ROUGE-2 | 24.22 | ROUGE-2 (%) | 57.1 |
| MHGraphBench - Two-hop Selection with Evidence | 38.42 | Accuracy (%) on the 1,200 two-hop selection items with sanit | 57.1 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 51.61 | MC2 (%) (0-shot) | 52.6 |
| Open Korean LLM - KorNAT-Helpful | 46.45 | Normalized accuracy (%) | 46.5 |
| IndoNutriQA - 0-shot ROUGE-L | 29.21 | ROUGE-L (%) | 42.9 |
| MHGraphBench - Entity Typing | 29.24 | Accuracy (%) on the 1,847 five-way entity typing items, lett | 42.9 |
| Open LLM Leaderboard v1 - HellaSwag | 79.09 | Normalized accuracy (%) (10-shot) | 36.7 |
| Open LLM Leaderboard v1 - MMLU | 55.56 | Accuracy (%) (5-shot) | 36.6 |
Interactive version: theaggregate.ai/model?slug=biomistral-7b · How It Works · Data refreshed daily, snapshot 2026-11-08.