Hard12: leaderboard
Single models in a coding harness attempt 12 hard, hand-checked tasks drawn from Terminal-Bench 4, DeepSWE, SWE-rebench, SWE-bench Pro V2 and Longtail-SWE, three runs each; the score is the share of the 36 runs solved. Unlike those source benchmarks it is one person's hand-picked hard subset, and the same page's multi-model setups are not admitted.
Metric: pass@1: mean solve rate over 12 hard coding tasks x 3 runs (%). Source: ibragim.dev. Saturation forecast: Around February 2027. 17 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6.1 Terra (Medium) | 83.3 | #20 (GPT-6.1 Terra) |
| 2 | Claude Opus 5.5 (Medium) | 83.3 | #5 (Claude Opus 5.5) |
| 3 | GPT-6.1 Sol (High) | 80.6 | #10 (GPT-6.1 Sol) |
| 4 | GPT-6.1 Sol (Medium) | 80.6 | #10 (GPT-6.1 Sol) |
| 5 | Kimi K3.1 Pro | 75 | #29 |
| 6 | GPT-6.1 Sol (Low) | 72.2 | #10 (GPT-6.1 Sol) |
| 7 | DeepSeek V4.1 Flash (1017) | 63.9 | #39 |
| 8 | Claude Opus 5.5 (Low) | 63.9 | #5 (Claude Opus 5.5) |
| 9 | Kimi K3 | 61.1 | #54 |
| 10 | GPT-6 Luna | 61.1 | #95 |
| 11 | Claude Haiku 5.5 | 50 | #65 |
| 12 | DeepSeek V4.1 Flash | 44.4 | #59 |
| 13 | Qwen 3.8 27B | 19.4 | #204 |
| 14 | DeepSeek V4 Flash (0731) | 16.7 | #139 |
| 15 | Beam 1.0 | 11.1 | #199 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #6 GPT-6 Astra.
Interactive version: theaggregate.ai/benchmark?slug=hard12 · How It Works · Data refreshed daily, snapshot 2026-11-13.