Claude 3.5 Sonnet (20241022): benchmark results
October 22, 2024 Claude 3.5 Sonnet snapshot, tracked when sources report the dated API model. Provider: Anthropic. Released 2024-10-22. Access: API.
Aggregate Rating 1659 ± 5, rank #291 of 1656 rated models, from 2009 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| AgentDojo | 85.69 | Robustness Score (%) | 100 |
| Aider Refactoring Benchmark | 92.1 | Percent completed correctly (imported) | 100 |
| BALROG BabyAI (VLM) | 82 | Progress (%) | 100 |
| BALROG MiniHack (VLM) | 22.5 | Progress (%) | 100 |
| BrowserGym WebLINX | 13.7 | Turn-level score (%, GenericAgent unless named) | 100 |
| CipherBank | 45.14 | Decryption Accuracy (%; 3-shot, mean of nine ciphers) | 100 |
| GitTaskBench (RepoMaster) | 62.96 | Task pass rate (%) | 100 |
| GitTaskBench (RepoMaster) - Execution Completion | 75.92 | Execution completion rate (%) | 100 |
| HELM AIR-Bench 2024 - #10.14: Torture/abuse | 96.67 | Refusal Rate (%) | 100 |
| HELM AIR-Bench 2024 - #10.16: Activities meant to kill | 80 | Refusal Rate (%) | 100 |
| HELM AIR-Bench 2024 - #10: Depicting Violence | 90.67 | Refusal Rate (%) | 100 |
| HELM AIR-Bench 2024 - #13: Harassment | 98.66 | Refusal Rate (%) | 100 |
Interactive version: theaggregate.ai/model?slug=claude-3-5-sonnet-20241022 · How It Works · Data refreshed daily, snapshot 2026-10-20.