Themis-CodeRewardBench - Functional Correctness: leaderboard
The functional-correctness split of Themis-CodeRewardBench (Paul, Glavaš and Gurevych, 2026): 4,778 code preference pairs from bug-fix commits, HumanEvalPack, MBPPPlusFix-Hard, MDEval, DebugEval and RunBugRun, where the preferred response is the correct one. Preference accuracy of pointwise reward-model scores in percent; higher is better.
Metric: Preference accuracy (%; share of the benchmark's preference pairs in which the reward model scores the preferred code response above the rejected one, pointwise and reference-free; Functional Correctness criterion, 4,778 pairs: commit bug fixes, HumanEvalPack, MBPPPlusFix-Hard, MDEval, DebugEval and RunBugRun-V1). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 50 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Themis-RM-32B | 94.27 | |
| 2 | Themis-RM-14B | 92.74 | |
| 3 | Themis-RM-8B | 91.75 | |
| 4 | Themis-RM-4B | 89.72 | |
| 5 | Llama-3.3-Nemotron-70B-Reward | 88.76 | |
| 6 | Athene-RM-70B | 87.69 | |
| 7 | CodeScaler-8B | 87.42 | |
| 8 | Skywork-Reward-V2-Qwen3-8B | 87.25 | |
| 9 | Skywork-Reward-V2-Qwen3-4B | 86.4 | |
| 10 | Llama-3.1-70B-Instruct-RM-RB2 | 86.23 |
Interactive version: theaggregate.ai/benchmark?slug=themis-coderewardbench-functional-correctness · How It Works · Data refreshed daily, snapshot 2026-11-12.