PaLM-2 (Unicorn): benchmark results
Google's largest PaLM 2 size tier (above Gecko, Otter, and Bison), announced at I/O in May 2023 and served as Vertex AI's text-unicorn API from November 2023. Provider: Google. Released 2023-11-30. Access: API.
Unified ELO 1510 ± 1, rank #650 of 1392 rated models, from 9 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| HELM WMT 2014 | 25.95 | BLEU-4 (%) | 98.9 |
| HELM v2 Lite - LegalBench | 70.25 | Exact Match (%) | 90 |
| HELM v2 Lite - MedQA | 76.92 | Exact Match (%) | 87.5 |
| HELM NaturalQuestions (Closed) | 43.48 | F1 (%) | 78.9 |
| HELM Lite | 68.11 | Mean win rate (self-reported) | 73.7 |
| HELM (Stanford) | 64.36 | Mean Win Rate (%) | 68.9 |
| HELM NaturalQuestions (Open) | 67.42 | F1 (%) | 31.1 |
| HELM v2 Lite - HumanEval (Code) | 0 | Pass@1 (%) | 11.8 |
| HELM NarrativeQA | 58.29 | F1 (%) | 11.1 |
Interactive version: theaggregate.ai/model?slug=palm-2-unicorn · How It Works · Data refreshed daily, snapshot 2026-09-05.