Lumimaid-v0.2-12B: benchmark results
NeverSleep's (Undi & IkariDev) roleplay fine-tune of Mistral Nemo Instruct 2407, v0.2 retrained on a heavily cleaned and deduplicated RP dataset. Provider: NeverSleep. Released 2024-07-25. Access: Open.
Aggregate Rating 1491 ± 46, rank #2494 of 10916 rated models, from 10 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard - MuSR | 48.21 | Score | 95.3 |
| Open LLM Leaderboard - GPQA | 31.46 | Score | 71.3 |
| Open LLM Leaderboard - BBH | 53.96 | Score | 68.6 |
| Open LLM Leaderboard - MMLU-Pro | 35.11 | Score | 51.8 |
| UGI - Writing | 31.95 | Writing Score | 43.3 |
| UGI - Willingness (W/10) | 5.2 | W/10 Score | 43.1 |
| Open LLM Leaderboard - MATH Level 5 | 5.66 | Score | 31.2 |
| UGI - Natural Intelligence | 16.68 | NatInt Score | 26.6 |
| UGI Leaderboard | 25.01 | UGI Score | 18.6 |
| Open LLM Leaderboard - IFEval | 10.99 | Score | 2 |
Interactive version: theaggregate.ai/model?slug=lumimaid-v0-2-12b · How It Works · Data refreshed daily, snapshot 2026-11-08.