karakuri-lm-8x7B-chat-v0.1: benchmark results
Provider: Other. Access: Open.
Aggregate Rating 1484 ± 30, rank #2612 of 10916 rated models, from 58 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Japanese LLM - SUM | 13.08 | Score (%) | 95.2 |
| Open Japanese LLM - Xlsum JA Rouge2 | 13.06 | Score (%) | 95.2 |
| Open Japanese LLM - Xlsum JA RougeLsum | 29.23 | Score (%) | 94.2 |
| Open Japanese LLM - Xlsum JA Bert Score JA F1 | 71.4 | Score (%) | 94.1 |
| Open Japanese LLM - Xlsum JA Rouge1 | 34.83 | Score (%) | 92.5 |
| Open Japanese LLM - Xlsum JA Bleu JA | 3.07 | Score (%) | 83.1 |
| pfgen-bench - Completion Mode - Fluency | 0.77 | Fluency Score | 83.1 |
| Open Japanese LLM - Wiki NER SET F1 | 11.5 | Score (%) | 82.8 |
| pfgen-bench - Completion Mode - Truthfulness | 0.87 | Truthfulness Score | 81.8 |
| Open Japanese LLM - AIO Char F1 | 68.57 | Score (%) | 81 |
| pfgen-bench - Completion Mode - Score | 0.63 | pfgen Score (mean of three) | 79.6 |
| Open Japanese LLM - Mbpp Pylint Check | 91.57 | Score (%) | 79.2 |
Interactive version: theaggregate.ai/model?slug=karakuri-lm-8x7b-chat-v0-1 · How It Works · Data refreshed daily, snapshot 2026-11-08.