Palmyra X V2 (33B): benchmark results
Writer's API-only 33B Palmyra-X V2 model (December 2023), pretrained on over 2T tokens and a strong HELM Lite performer for its size. Provider: Writer. Released 2023-12-01. Access: API.
Aggregate Rating 1494 ± 26, rank #2447 of 10916 rated models, from 31 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| HELM Lite - WMT 2014 (de-en) | 22.65 | BLEU-4 (%) | 96.7 |
| HELM Lite - WMT 2014 (hi-en) | 19.96 | BLEU-4 (%) | 95.6 |
| HELM WMT 2014 | 23.88 | BLEU-4 (%) | 95.6 |
| HELM Lite - WMT 2014 (cs-en) | 25.48 | BLEU-4 (%) | 94.4 |
| HELM Lite - LegalBench - PROA | 98.95 | Exact Match (%) | 92.2 |
| HELM Lite - WMT 2014 (ru-en) | 27.03 | BLEU-4 (%) | 92.2 |
| HELM Lite - WMT 2014 (fr-en) | 24.29 | BLEU-4 (%) | 92.1 |
| HELM v2 Lite - LegalBench | 61.37 | Exact Match (%) | 80 |
| HELM NaturalQuestions (Open) | 75.15 | F1 (%) | 77.8 |
| HELM NaturalQuestions (Closed) | 42.81 | F1 (%) | 74.4 |
| HELM Lite | 63.98 | Mean win rate (imported) | 71.1 |
| HELM Lite - LegalBench - Corporate Lobbying | 81.22 | Exact Match (%) | 70 |
Interactive version: theaggregate.ai/model?slug=palmyra-x-v2-33b · How It Works · Data refreshed daily, snapshot 2026-11-08.