Sarvam M: benchmark results
Sarvam AI's 24B open-weights hybrid-reasoning model (May 2025), built on Mistral Small with SFT and RLVR for Indian languages, math and code. Provider: Sarvam AI. Released 2025-05-23. Access: Open.
Aggregate Rating 1539 ± 23, rank #643 of 1635 rated models, from 16 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| IndicQE-APE - Segment-Level Quality Estimation | 0.44 | Macro Spearman correlation with human direct assessment (-1 | 80 |
| VakQA (English, Indic MT) | 2.39 | Judge score (1-5) (2,001 Telugu spoken factoid questions fro | 71.4 |
| VakQA (English, Seamless MT) | 2.29 | Judge score (1-5) (2,001 Telugu spoken factoid questions fro | 71.4 |
| VakQA | 1.98 | Judge score (1-5) (2,001 Telugu spoken factoid questions fro | 57.1 |
| Kinship Term Generation | 40.48 | Accepted-term accuracy (%; birthday greetings and wedding in | 50 |
| Kinship Term Generation - Birthday Greetings | 41.13 | Accepted-term accuracy (%; short birthday greeting to the sp | 50 |
| Kinship Term Generation - Wedding Invitations | 39.83 | Accepted-term accuracy (%; short wedding invitation to the s | 50 |
| VakQA (IndicWhisper ASR Transcripts) | 1.69 | Judge score (1-5) (2,001 Telugu spoken factoid questions fro | 25 |
| VakQA (Seamless FT ASR Transcripts) | 1.75 | Judge score (1-5) (2,001 Telugu spoken factoid questions fro | 25 |
| HindiGen | 22.92 | 3C3H Score (%) | 17.6 |
| Kaggle IndicGenBench - XQuAD-In | 51.48 | Token F1 | 16.7 |
| Kaggle IndicGenBench - XorQA-In | 54.64 | Token F1 | 9.1 |
Interactive version: theaggregate.ai/model?slug=sarvam-m · How It Works · Data refreshed daily, snapshot 2026-11-05.