MamayLM-Gemma-3-12B-IT-v1.0: benchmark results
INSAIT's open Ukrainian-focused multimodal Gemma 3 12B fine-tune, released in September 2025. Provider: INSAIT. Released 2025-09-25. Access: Open.
Aggregate Rating 1560 ± 27, rank #584 of 1656 rated models, from 184 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Ukrainian LLM - Global MMLU Full UK Human Sexuality | 80.92 | Score (%) | 100 |
| Ukrainian LLM - Global MMLU Full UK Virology | 56.63 | Score (%) | 100 |
| EuroEval Ukrainian NLU - Cross Domain UK Reviews | 64.08 | Matthews correlation coefficient (%) | 98 |
| EuroEval Croatian NLU - MMS HR | 47.14 | Matthews correlation coefficient (%) | 97.6 |
| EuroEval Serbian NLU - MMS SR | 51.48 | Matthews correlation coefficient (%) | 97.5 |
| EuroEval Slovak NLU - CSFD Sentiment SK | 62.23 | Matthews correlation coefficient (%) | 94.6 |
| EuroEval Ukrainian NLU - ScaLA UK | 42.16 | Matthews correlation coefficient (%) | 92.5 |
| EuroEval Czech NLU - CSFD Sentiment | 62.56 | Matthews correlation coefficient (%) | 91.7 |
| Ukrainian LLM - Global MMLU Full UK Abstract Algebra | 41 | Score (%) | 91.7 |
| Ukrainian LLM - Global MMLU Full UK High School US History | 86.27 | Score (%) | 91.7 |
| Ukrainian LLM - Global MMLU Full UK Marketing | 87.61 | Score (%) | 91.7 |
| Ukrainian LLM - Global MMLU Full UK Medical Genetics | 83 | Score (%) | 91.7 |
Interactive version: theaggregate.ai/model?slug=mamaylm-gemma-3-12b-it-v1-0 · How It Works · Data refreshed daily, snapshot 2026-10-20.