Orchid-BCB-Expand Original Pass@1: leaderboard
Orchid-BCB-Expand Original Pass@1 — Randomsamplingtemperature0.8,maxlength1024tokens; HumanEval+ protocol forHEval,BigCodeBench protocol forBCB/BCBExpand. Explicitthree subsets164/164/976 originaltasks. Pass@1/3/5 distinct. Table3 explicitCodeLlama34B,Qwen2.5Coder32B,DeepSeekV3Dec2024,DeepSeekR1Jan2025,GPT4Mar2023,Claude3.5Jun2024 withAnthropicClaude3.5Sonnetcitation. GenericGPT4API snapshot andCodeLlama/Qwenbase-vs-instruct notexplicitlyspecified: retainnative labels/versionunknown,do notinfer checkpointaliases. Metric: Pass@1 (%). Source: https://arxiv.org/html/2604.21505v1
Metric: Pass@1 (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 16 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | DeepSeek V3 | 50.04 | #722 | |
| 2 | GPT-4 | 47.62 | #976 | |
| 3 | Claude 3.5 Sonnet | 45.16 | #750 | |
| 4 | Qwen-2.5-Coder | Orchid run: evaluated-size=32B | 44.67 | |
| 5 | DeepSeek R1 | 31.42 | #478 | |
| 6 | CodeLlama | Orchid run: evaluated-size=34B | 5.72 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=orchid-bcb-expand-original-pass-1 · How It Works · Data refreshed daily, snapshot 2026-11-16.