Gemmasutra-9B-v1: benchmark results
TheDrummer's roleplay-focused community variant of Gemma 2 9B, tuned for uninhibited creative output. Provider: BeaverAI. Released 2024-07-17. Access: Open.
Aggregate Rating 1528 ± 19, rank #689 of 1656 rated models, from 25 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-EQ-Bench | 60.3 | EQ-Bench score | 99.3 |
| Open Korean LLM Leaderboard | 49.67 | Average Score (%) | 97.1 |
| Open Ko-LLM Leaderboard | 49.67 | Average (%) | 97 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 63.23 | Exact match, flexible extract (%) | 95.9 |
| Open LLM Leaderboard - MuSR | 48.46 | Score | 95.7 |
| Open Korean LLM - Ko-IFEval | 52.46 | Strict accuracy, prompt/instruction mean (%) | 95.2 |
| Open Korean LLM - KorNAT-CKA | 31.47 | Normalized accuracy (%) | 86.9 |
| Open Portuguese LLM - ASSIN2 RTE | 93 | Macro F1 (%) | 85.5 |
| Open Portuguese LLM - ENEM | 73.48 | Accuracy (%) | 85 |
| Open Portuguese LLM - FaQuAD NLI | 78.42 | Macro F1 (%) | 84 |
| Open Portuguese LLM - BLUEX | 63.56 | Accuracy (%) | 83.7 |
| Open Portuguese LLM - OAB Exams | 52.26 | Accuracy (%) | 81.5 |
Interactive version: theaggregate.ai/model?slug=gemmasutra-9b-v1 · How It Works · Data refreshed daily, snapshot 2026-10-20.