Palmyra X V3 (72B): benchmark results
Writer's 72B enterprise API model, the top non-OpenAI performer on Stanford's HELM Lite leaderboard at its debut. Provider: Writer. Released 2024-01-01. Access: API.
Aggregate Rating 1551 ± 22, rank #1698 of 10916 rated models, from 89 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| HELM Lite - WMT 2014 (cs-en) | 27.51 | BLEU-4 (%) | 100 |
| HELM Lite - WMT 2014 (fr-en) | 27.71 | BLEU-4 (%) | 100 |
| HELM Lite - WMT 2014 (hi-en) | 23.83 | BLEU-4 (%) | 100 |
| HELM WMT 2014 | 26.19 | BLEU-4 (%) | 100 |
| HELM Lite - WMT 2014 (de-en) | 23.48 | BLEU-4 (%) | 98.9 |
| HELM Lite - WMT 2014 (ru-en) | 28.4 | BLEU-4 (%) | 98.9 |
| HELM v2 Lite - LegalBench | 70.66 | Exact Match (%) | 95 |
| HELM MMLU - Human Sexuality | 92.37 | EM | 94.9 |
| HELM MMLU - Us Foreign Policy | 96 | EM | 93.6 |
| HELM MMLU - College Biology | 95.14 | EM | 91.7 |
| HELM MMLU - Business Ethics | 83 | EM | 87.8 |
| HELM v2 Lite - MedQA | 76.92 | Exact Match (%) | 87.5 |
Interactive version: theaggregate.ai/model?slug=palmyra-x-v3-72b · How It Works · Data refreshed daily, snapshot 2026-11-08.