Palmyra Med: benchmark results
Writer's 70B medical-domain specialist tuned on biomedical data, released July 2024 alongside Palmyra Fin. Provider: Writer. Released 2024-07-31. Access: API.
Aggregate Rating 1465 ± 39, rank #3026 of 10916 rated models, from 378 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| HELM AIR-Bench 2024 - #6.26: Government services | 13.33 | Refusal Rate (%) | 95.9 |
| PubMedQA | 81.1 | Accuracy (%) | 87 |
| HELM AIR-Bench 2024 - #6.25: Financial | 16.67 | Refusal Rate (%) | 84.9 |
| HELM AIR-Bench 2024 - #17.2: Suggestive | 70 | Refusal Rate (%) | 83.7 |
| HELM AIR-Bench 2024 - #39-40.55: Classification of individuals - Health conditions | 100 | Refusal Rate (%) | 82.6 |
| HELM AIR-Bench 2024 - #23.1: Suicide | 100 | Refusal Rate (%) | 79.1 |
| HELM AIR-Bench 2024 - #23.3: Eating Disorders (anorexia/bulimia) | 100 | Refusal Rate (%) | 79.1 |
| HELM AIR-Bench 2024 - #2.7: Malware | 100 | Refusal Rate (%) | 77.9 |
| HELM AIR-Bench 2024 - #39-40.58: Classification of individuals - Genetic Information | 93.33 | Refusal Rate (%) | 77.9 |
| HELM AIR-Bench 2024 - #39-40.34: Characterization of identity - Predicted Personality | 100 | Refusal Rate (%) | 76.7 |
| HELM AIR-Bench 2024 - #14.14: Color | 100 | Refusal Rate (%) | 74.4 |
| HELM AIR-Bench 2024 - #14.21: Caste | 100 | Refusal Rate (%) | 74.4 |
Interactive version: theaggregate.ai/model?slug=palmyra-med · How It Works · Data refreshed daily, snapshot 2026-11-08.