Orchid-BCB-Expand Original Pass@3: leaderboard

Orchid-BCB-Expand Original Pass@3 — Randomsamplingtemperature0.8,maxlength1024tokens; HumanEval+ protocol forHEval,BigCodeBench protocol forBCB/BCBExpand. Explicitthree subsets164/164/976 originaltasks. Pass@1/3/5 distinct. Table3 explicitCodeLlama34B,Qwen2.5Coder32B,DeepSeekV3Dec2024,DeepSeekR1Jan2025,GPT4Mar2023,Claude3.5Jun2024 withAnthropicClaude3.5Sonnetcitation. GenericGPT4API snapshot andCodeLlama/Qwenbase-vs-instruct notexplicitlyspecified: retainnative labels/versionunknown,do notinfer checkpointaliases. Metric: Pass@3 (%). Source: https://arxiv.org/html/2604.21505v1

Metric: Pass@3 (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 16 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1DeepSeek V358.26#722
2GPT-456.71#976
3Qwen-2.5-CoderOrchid run: evaluated-size=32B50.91
4Claude 3.5 Sonnet49.48#750
5DeepSeek R143.1#478
6CodeLlamaOrchid run: evaluated-size=34B14.4

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=orchid-bcb-expand-original-pass-3 · How It Works · Data refreshed daily, snapshot 2026-11-16.