Orchid-HEval Original Pass@5: leaderboard
Orchid-HEval Original Pass@5 — Randomsamplingtemperature0.8,maxlength1024tokens; HumanEval+ protocol forHEval,BigCodeBench protocol forBCB/BCBExpand. Explicitthree subsets164/164/976 originaltasks. Pass@1/3/5 distinct. Table3 explicitCodeLlama34B,Qwen2.5Coder32B,DeepSeekV3Dec2024,DeepSeekR1Jan2025,GPT4Mar2023,Claude3.5Jun2024 withAnthropicClaude3.5Sonnetcitation. GenericGPT4API snapshot andCodeLlama/Qwenbase-vs-instruct notexplicitlyspecified: retainnative labels/versionunknown,do notinfer checkpointaliases. Metric: Pass@5 (%). Source: https://arxiv.org/html/2604.21505v1
Metric: Pass@5 (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 16 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude 3.5 Sonnet | 92.07 | #750 | |
| 2 | DeepSeek R1 | 90.24 | #478 | |
| 3 | GPT-4 | 86.59 | #976 | |
| 4 | DeepSeek V3 | 84.76 | #722 | |
| 5 | Qwen-2.5-Coder | Orchid run: evaluated-size=32B | 70.73 | |
| 6 | CodeLlama | Orchid run: evaluated-size=34B | 43.29 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=orchid-heval-original-pass-5 · How It Works · Data refreshed daily, snapshot 2026-11-16.