BigCodeReward: leaderboard

Each model acts as a judge on all 4,731 BigCodeArena conversations: given the user prompts, two models' code responses and their execution outputs (logs and screenshots), it says which response is better or that they tie. The score is agreement with the human vote; unlike reward benchmarks on text-only code, the judge sees what the code did when run.

Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Not forecast. 12 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1Qwen 2.5 VL 72B Instruct66.2#849
2GPT-4o Mini (2024-07-18)64.5#1494
3Claude Sonnet 4 (20250514)62.3#405
4Claude 3.7 Sonnet (20250219)62.2#421
5Qwen 2.5 VL 32B Instruct61.9#1074
6Gemma 3 27B (IT)61.1#1339
7GPT-4.1 (2025-04-14)60#501
8GLM-4.5V55.2#758
9MiMo-VL-7B-RL50.7#1162
10InternVL3-38B48#927
11InternVL3-78B46.8#755
12Kimi-VL-A3B-Thinking-250646.2#1599

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=bigcodereward · How It Works · Data refreshed daily, snapshot 2026-11-17.