Lumimaid-v0.2-12B: benchmark results
NeverSleep's (Undi & IkariDev) roleplay fine-tune of Mistral Nemo Instruct 2407, v0.2 retrained on a heavily cleaned and deduplicated RP dataset. Provider: Other. Released 2024-07-25. Access: Open.
Unified ELO 1483 ± 1, rank #774 of 1392 rated models, from 10 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard - MuSR | 21.3 | Score | 96.9 |
| Open LLM Leaderboard - GPQA | 8.61 | Score | 71.4 |
| Open LLM Leaderboard - BBH | 34.41 | Score | 68.3 |
| Open LLM Leaderboard - MMLU-Pro | 27.91 | Score | 51.8 |
| UGI - Writing | 31.95 | Writing Score | 44.1 |
| UGI - Willingness (W/10) | 5.2 | W/10 Score | 42 |
| Open LLM Leaderboard - MATH Level 5 | 5.66 | Score | 31.3 |
| UGI - Natural Intelligence | 16.68 | NatInt Score | 27.1 |
| UGI Leaderboard | 25.01 | UGI Score | 19 |
| Open LLM Leaderboard - IFEval | 10.99 | Score | 2 |
Interactive version: theaggregate.ai/model?slug=lumimaid-v0-2-12b · How It Works · Data refreshed daily, snapshot 2026-09-05.