Gemma-2-Llama-Swallow-27B-pt-v0.1: benchmark results
Provider: Google. Access: API.
Aggregate Rating 1554 ± 38, rank #1644 of 10916 rated models, from 22 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Swallow - Pre-trained Japanese - WMT20 En-Ja | 32.1 | BLEU | 98 |
| Swallow - Pre-trained Japanese - Average | 59.4 | Average Score (%) | 95.9 |
| Swallow - Pre-trained Japanese - JEMHopQA | 66 | Character F1 (%) | 95.9 |
| Swallow - Pre-trained Japanese - NIILC | 67.1 | Character F1 (%) | 93.9 |
| Swallow - Pre-trained Japanese - WMT20 Ja-En | 25.5 | BLEU | 92.9 |
| Swallow - Pre-trained English - HumanEval | 65.8 | Pass@1 (%) | 91.8 |
| Swallow - Pre-trained Japanese - JCommonsenseQA | 95.8 | Accuracy (%) | 91.8 |
| Swallow - Pre-trained Japanese - JHumanEval | 62.9 | Pass@1 (%) | 91.8 |
| Swallow - Pre-trained English - TriviaQA | 75.6 | Exact Match (%) | 89.8 |
| Swallow - Pre-trained English - OpenBookQA | 41.4 | Accuracy (%) | 87.8 |
| Swallow - Pre-trained English - XWINO | 91.5 | Accuracy (%) | 87.8 |
| Swallow - Pre-trained Japanese - JSQuAD | 92.4 | Character F1 (%) | 86.7 |
Interactive version: theaggregate.ai/model?slug=gemma-2-llama-swallow-27b-pt-v0-1 · How It Works · Data refreshed daily, snapshot 2026-11-08.