KOR-Bench - Cipher: leaderboard
Encrypting and decrypting with 25 modified classical ciphers whose rules and keys are given in the prompt, so it measures applying an unfamiliar cipher, not breaking one. Chat models, zero-shot. Higher is better. Source: arxiv.org/abs/2410.06526.
Metric: Accuracy (%; zero-shot, 250 questions). Source: arxiv.org. Saturation forecast: Estimated already saturated. 38 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | O1 Preview (2024-09-12) | 82.8 | #220 |
| 2 | O1 Mini (2024-09-12) | 79.6 | #414 |
| 3 | GPT-4o (2024-05-13) | 42.8 | #338 |
| 4 | Claude 3.5 Sonnet (20240620) | 33.2 | #309 |
| 5 | Llama 3.1 405B Instruct | 31.2 | #472 |
| 6 | Qwen 2.5 32B Instruct | 26.8 | #511 |
| 7 | Qwen 2.5 72B Instruct | 26.4 | #470 |
| 8 | GPT-4 Turbo | 23.2 | #419 |
| 9 | Mistral Large 2 (Jul) | 22.8 | #498 |
| 10 | Yi Large | 20.8 | #565 |
| 11 | Llama 3.1 70B Instruct | 20.4 | #570 |
| 12 | Llama 3 70B Instruct | 20.4 | #643 |
| 13 | DeepSeek V2.5 | 18 | #503 |
| 14 | Qwen 2.5 14B Instruct | 14.4 | #657 |
| 15 | Qwen 2 72B Instruct | 12.8 | #584 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=kor-bench-cipher · How It Works · Data refreshed daily, snapshot 2026-11-07.