Claude 3 Sonnet (20240229): benchmark results

Claude 3 Sonnet (20240229) API snapshot, publicly available from 2024-03-04. The dated model ID identifies the snapshot; it is distinct from the public release date. Provider: Anthropic. Released 2024-03-04. Access: API.

Aggregate Rating 1531 ± 9, rank #677 of 1656 rated models, from 781 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
CRMArena - PVI (Function Calling Native)41.5Exact match (%)100
LLM Grid Game - connect-four / list / prompt 2024-06-12 / player 280Win rate versus random (%)100
LLM Grid Game - gomoku / image / prompt 2024-06-12 / player 2100Win rate versus random (%)100
HELM AIR-Bench 2024 - Political Usage97.33Refusal Rate (%)99.4
HELM VHELM - Blink Perception - Spatial Relation99.3Prefix Quasi-Exact Match (%)98.9
HELM AIR-Bench 2024 - #24: Political Persuasion93.33Refusal Rate (%)98.8
HELM AIR-Bench 2024 - #41-42.30: Unauthorized collection/gathering/stealing - Location data100Refusal Rate (%)98.8
HELM AIR-Bench 2024 - Criminal Activities100Refusal Rate (%)98.3
HELM VHELM - Blink Reasoning82PEM97.9
HELM AIR-Bench 2024 - #24.2: Generating high-volume campaign materials93.33Refusal Rate (%)97.7
HELM AIR-Bench 2024 - #27: Disrupting Social Order (China-unique)99.44Refusal Rate (%)97.7
HELM AIR-Bench 2024 - #39-40.6: Discrimination in employment, benefits, or services - Religion100Refusal Rate (%)97.7

Interactive version: theaggregate.ai/model?slug=claude-3-sonnet-20240229 · How It Works · Data refreshed daily, snapshot 2026-10-20.