Mistral Large 2 (Jul): benchmark results

July 2024 release of Mistral AI's Large 2, a 123B model with downloadable instruct weights under the Mistral Research License. Provider: Mistral. Released 2024-07-24. Access: API.

Aggregate Rating 1587 ± 7, rank #484 of 1656 rated models, from 817 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
NVIDIA LLM Robustness - MATH Accuracy71.14Accuracy (%)100
NVIDIA LLM Robustness - MATH Consistency67.71Consistency (%)100
OmniToM - Belief Labeling - Context92.97Belief-labeling accuracy (%): exact-match accuracy of the pr100
OmniToM - Belief Labeling - Mental Source86.32Belief-labeling accuracy (%): exact-match accuracy of the pr100
OmniToM - Belief Labeling - Order97.25Belief-labeling accuracy (%): exact-match accuracy of the pr100
OmniToM - Belief Labeling - Truth Status86.53Belief-labeling accuracy (%): exact-match accuracy of the pr100
Open PL LLM - PolEmo2-IN (multiple choice, 5-shot)91.69Accuracy (%)100
Open PL LLM - PolEmo2-OUT (multiple choice, 5-shot)83.4Accuracy (%)100
OpenCompass Knowledge - Science - Chinese (CompassBench 2407)77.6Score (%)100
OpenCompass LLM - Math (CompassBench 2407)72.8Score (%)100
OpenCompass Math - College - Chinese (CompassBench 2407)72Score (%)100
OpenCompass Math - College - English (CompassBench 2407)69.3Score (%)100

Interactive version: theaggregate.ai/model?slug=mistral-large-2-jul · How It Works · Data refreshed daily, snapshot 2026-10-20.