OpenChat-3.5-0106-Gemma: benchmark results
OpenChat's Gemma-based 3.5-0106 model, trained with their C-RLFT method on the openchat-3.5-0106 data, unlike their usual Mistral-based releases. Provider: OpenChat. Released 2024-03-09. Access: Open.
Aggregate Rating 1421 ± 13, rank #1123 of 1635 rated models, from 58 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard v1 - GSM8K | 72.86 | Accuracy (%) (5-shot) | 98.2 |
| Open Portuguese LLM - FaQuAD NLI | 84.26 | Macro F1 (%) | 97.8 |
| Open Portuguese LLM - ASSIN2 RTE | 93.99 | Macro F1 (%) | 93.8 |
| Open PL LLM - PoQuAD Open Book (generative, 5-shot) | 70.28 | Levenshtein Similarity (%) | 85.9 |
| Open PL LLM - DYK (generative, 5-shot) | 68.28 | Binary F1 (%) | 85.8 |
| Open PL LLM - EQ-Bench | 55.55 | EQ-Bench Score | 84.6 |
| Open PL LLM - PolQA Reranking (multiple choice, 5-shot) | 80.06 | Accuracy (%) | 83 |
| Open PL LLM - DYK (multiple choice, 5-shot) | 67.35 | Binary F1 (%) | 82.8 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 46.4 | Exact match, flexible extract (%) | 80.8 |
| Open PL LLM - RAG | 70.28 | Average RAG Score (%) | 79 |
| Open PL LLM - PPC (multiple choice, 5-shot) | 77.1 | Accuracy (%) | 75.4 |
| Open LLM Leaderboard v1 - MMLU | 64.69 | Accuracy (%) (5-shot) | 75 |
Interactive version: theaggregate.ai/model?slug=openchat-3-5-0106-gemma · How It Works · Data refreshed daily, snapshot 2026-11-05.