Claude Sonnet 4 (20250514): benchmark results
May 14, 2025 Claude Sonnet 4 snapshot, tracked when sources report the dated API model. Provider: Anthropic. Released 2025-05-22. Access: API.
Aggregate Rating 1692 ± 8, rank #213 of 1656 rated models, from 640 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| CausalVerify - Event Study | 87.5 | Execution-grounded pass rate (%; L2b+ over the 24 event-stud | 100 |
| Galileo Agent - Healthcare TSQ | 95 | Task Success Quality (%) | 100 |
| GraphInstruct (Generation) - L1 Single Constraint | 98.3 | Level quality score (0-100; the paper's 0-1 score times 100) | 100 |
| HELM AIR-Bench 2024 - #39-40.57: Classification of individuals - Pregnancy Status | 86.67 | Refusal Rate (%) | 100 |
| RealBench (Holistic) - Level 4 | 14.3 | Test pass rate (%), passed tests over all tests of each task | 100 |
| RealBench (Incremental) - Level 2 | 10.75 | Test pass rate (%), passed tests over all tests of each task | 100 |
| RealBench (RAG) - Level 3 | 19.49 | Test pass rate (%), passed tests over all tests of each task | 100 |
| RealBench (RAG) - Level 4 | 21.33 | Test pass rate (%), passed tests over all tests of each task | 100 |
| SAHARA - News Classification | 40.09 | Macro-F1 (%) | 100 |
| Vul4Py (Direct Prompting) | 5 | Plausible patch rate (%; share of the 100 real Python vulner | 100 |
| HELM AIR-Bench 2024 - #39-40.35: Characterization of identity - Health conditions | 91.67 | Refusal Rate (%) | 99.4 |
| T2S-Bench - Boundary Testing | 84.17 | Exact match (%) on the 120 Boundary Testing questions on T2S | 98.9 |
Interactive version: theaggregate.ai/model?slug=claude-sonnet-4-20250514 · How It Works · Data refreshed daily, snapshot 2026-10-20.