Claude 3.5 Sonnet (20241022): benchmark results
October 22, 2024 Claude 3.5 Sonnet snapshot, tracked when sources report the dated API model. Provider: Anthropic. Released 2024-10-22. Access: API.
Unified ELO 1600 ± 1, rank #264 of 1392 rated models, from 1031 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Aider Refactoring Benchmark | 92.1 | Percent completed correctly (self-reported) | 100 |
| BALROG BabyAI (VLM) | 82 | Progress (%) | 100 |
| BALROG MiniHack (VLM) | 22.5 | Progress (%) | 100 |
| HELM MedHELM - MedAlign | 4.26 | Jury Score | 100 |
| HELM MedHELM - MedHallu | 92.6 | EM | 100 |
| MEGA-Bench Task - AV View Identification | 33.3 | Task Score (%) | 100 |
| MEGA-Bench Task - Ball Cup Swap 3 | 64.3 | Task Score (%) | 100 |
| MEGA-Bench Task - Code Translation Easy | 78.6 | Task Score (%) | 100 |
| MEGA-Bench Task - Constrained Generation Contain Repeat Length | 46.7 | Task Score (%) | 100 |
| MEGA-Bench Task - Counting | 85.7 | Task Score (%) | 100 |
| MEGA-Bench Task - Doc VQA | 92.5 | Task Score (%) | 100 |
| MEGA-Bench Task - Geometry Reasoning Circled Letter | 96.4 | Task Score (%) | 100 |
Interactive version: theaggregate.ai/model?slug=claude-3-5-sonnet-20241022 · How It Works · Data refreshed daily, snapshot 2026-09-05.