Orchid-HEval Original Pass@5: leaderboard

Orchid-HEval Original Pass@5 — Randomsamplingtemperature0.8,maxlength1024tokens; HumanEval+ protocol forHEval,BigCodeBench protocol forBCB/BCBExpand. Explicitthree subsets164/164/976 originaltasks. Pass@1/3/5 distinct. Table3 explicitCodeLlama34B,Qwen2.5Coder32B,DeepSeekV3Dec2024,DeepSeekR1Jan2025,GPT4Mar2023,Claude3.5Jun2024 withAnthropicClaude3.5Sonnetcitation. GenericGPT4API snapshot andCodeLlama/Qwenbase-vs-instruct notexplicitlyspecified: retainnative labels/versionunknown,do notinfer checkpointaliases. Metric: Pass@5 (%). Source: https://arxiv.org/html/2604.21505v1

Metric: Pass@5 (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 16 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Claude 3.5 Sonnet92.07#750
2DeepSeek R190.24#478
3GPT-486.59#976
4DeepSeek V384.76#722
5Qwen-2.5-CoderOrchid run: evaluated-size=32B70.73
6CodeLlamaOrchid run: evaluated-size=34B43.29

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=orchid-heval-original-pass-5 · How It Works · Data refreshed daily, snapshot 2026-11-16.