Orchid-HEval Original Pass@1: leaderboard

Orchid-HEval Original Pass@1 — Randomsamplingtemperature0.8,maxlength1024tokens; HumanEval+ protocol forHEval,BigCodeBench protocol forBCB/BCBExpand. Explicitthree subsets164/164/976 originaltasks. Pass@1/3/5 distinct. Table3 explicitCodeLlama34B,Qwen2.5Coder32B,DeepSeekV3Dec2024,DeepSeekR1Jan2025,GPT4Mar2023,Claude3.5Jun2024 withAnthropicClaude3.5Sonnetcitation. GenericGPT4API snapshot andCodeLlama/Qwenbase-vs-instruct notexplicitlyspecified: retainnative labels/versionunknown,do notinfer checkpointaliases. Metric: Pass@1 (%). Source: https://arxiv.org/html/2604.21505v1

Metric: Pass@1 (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 16 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1DeepSeek V381.22#722
2DeepSeek R177.68#478
3Claude 3.5 Sonnet76.71#750
4GPT-472.68#976
5Qwen-2.5-CoderOrchid run: evaluated-size=32B69.15
6CodeLlamaOrchid run: evaluated-size=34B33.66

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=orchid-heval-original-pass-1 · How It Works · Data refreshed daily, snapshot 2026-11-16.