Claude 3 Haiku (20240307): benchmark results

Claude 3 Haiku (20240307) API snapshot, publicly available from 2024-03-13. The dated model ID identifies the snapshot; it is distinct from the public release date. Provider: Anthropic. Released 2024-03-13. Access: API.

Aggregate Rating 1503 ± 8, rank #771 of 1656 rated models, from 787 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
HELM Safety - Anthropic Red Team99.96LM Evaluated Safety score100
SocialBias-Bench - Gender5.25Code bias score for gender (%; share of executable generated100
SocialBias-Bench - Race4.31Code bias score for race (%; share of executable generated s100
SocialBias-Bench - Religion5.48Code bias score for religion (%; share of executable generat100
HELM AIR-Bench 2024 - #20.8: Pornography100Refusal Rate (%)98.8
HELM AIR-Bench 2024 - #41-42.30: Unauthorized collection/gathering/stealing - Location data100Refusal Rate (%)98.8
HELM AIR-Bench 2024 - #11.17: Guns93.33Refusal Rate (%)98.3
HELM AIR-Bench 2024 - #11: Weapon Usage & Development98.89Refusal Rate (%)98.3
HELM AIR-Bench 2024 - #16.35: Vulgarity100Refusal Rate (%)98.3
HELM AIR-Bench 2024 - #1: Confidentiality100Refusal Rate (%)98.3
HELM AIR-Bench 2024 - #8.6: Denying or minimizing well-documented, major violent events or the victimhood of such events/Denying the deeds of martyrdom100Refusal Rate (%)98.3
HELM AIR-Bench 2024 - #21.2: Pedophilia100Refusal Rate (%)97.7

Interactive version: theaggregate.ai/model?slug=claude-3-haiku-20240307 · How It Works · Data refreshed daily, snapshot 2026-10-20.