Palmyra-X-004: benchmark results
Writer's enterprise frontier model emphasizing tool calling and built-in RAG for AI agents and workflows (October 2024). Provider: Writer. Released 2024-10-09. Access: API.
Aggregate Rating 1576 ± 26, rank #1320 of 10916 rated models, from 463 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| HELM MMLU - High School Computer Science | 95 | EM | 97.4 |
| HELM Lite - LegalBench - Abercrombie | 83.16 | Exact Match (%) | 97.2 |
| HELM MMLU - Abstract Algebra | 75 | EM | 94.9 |
| HELM MMLU - Human Sexuality | 92.37 | EM | 94.9 |
| HELM Lite - LegalBench | 72.95 | Exact Match (%) | 93.3 |
| HELM Lite - LegalBench - PROA | 98.95 | Exact Match (%) | 92.2 |
| HELM Lite | 85.05 | Mean win rate (imported) | 92.1 |
| HELM Safety - Xstest | 98.39 | LM Evaluated Safety score | 91.9 |
| HELM MMLU - Clinical Knowledge | 87.92 | EM | 90.4 |
| HELM MMLU - College Medicine | 80.92 | EM | 89.7 |
| HELM MMLU - Elementary Mathematics | 84.13 | EM | 88.5 |
| HELM Lite - MATH Level 1 - Counting and Probability | 89.74 | Equivalent (CoT) (%) | 87.8 |
Interactive version: theaggregate.ai/model?slug=palmyra-x-004 · How It Works · Data refreshed daily, snapshot 2026-11-08.