Gemma 3 12B (IT): benchmark results
Instruction-tuned Gemma 3 12B checkpoint. Provider: Google. Released 2025-03-12. Access: Open.
Aggregate Rating 1534 ± 4, rank #664 of 1656 rated models, from 2431 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Alexandria - English to Lebanese Arabic (Single Turn, Gender Only) | 21.3 | spBLEU (0-100; SacreBLEU with the FLORES-200 SentencePiece t | 100 |
| Alexandria - English to Lebanese Arabic (Single Turn, No Metadata) | 20.82 | spBLEU (0-100; SacreBLEU with the FLORES-200 SentencePiece t | 100 |
| BBC News Video Captioning - Entity Fidelity | 31.4 | Entity Fidelity Score (0-1, times 100): per-clip harmonic me | 100 |
| BBC News Video Captioning - Text Similarity | 43.87 | Text similarity (cosine, -1 to 1, times 100) between all-Min | 100 |
| BehaviorBench (Behavioral Traces) - Belief Confidence Error (DirectGen) | 0.11 | Confidence mean absolute error on a 0 to 1 scale: gap betwee | 100 |
| BehaviorBench (Behavioral Traces) - Trade Amount Error (DirectGen) | 8.34 | Median absolute error of the predicted trade amount (paper u | 100 |
| BehaviorBench (Behavioral Traces) - Trade Amount Error (RetrievalGen) | 8.29 | Median absolute error of the predicted trade amount (paper u | 100 |
| Belebele European - Albanian | 87.22 | Accuracy (%, 0-shot) | 100 |
| Belebele European - Estonian | 86 | Accuracy (%, 0-shot) | 100 |
| Belebele European - Greek | 89.11 | Accuracy (%, 0-shot) | 100 |
| Belebele European - Icelandic | 83.44 | Accuracy (%, 0-shot) | 100 |
| Belebele European - Lithuanian | 86.67 | Accuracy (%, 0-shot) | 100 |
Interactive version: theaggregate.ai/model?slug=gemma-3-12b-it · How It Works · Data refreshed daily, snapshot 2026-10-20.