Themis-CodeRewardBench: leaderboard
Reward-model evaluation on code (Themis-CodeRewardBench, Paul, Glavaš and Gurevych, 2026): 8,866 chosen-rejected code response pairs across functional correctness, execution efficiency, memory efficiency, readability and maintainability, and security hardness, in eight programming languages. Each response is scored pointwise without a reference; preference accuracy in percent over all pairs; higher is better.
Metric: Preference accuracy (%; share of the benchmark's preference pairs in which the reward model scores the preferred code response above the rejected one, pointwise and reference-free; all 8,866 pairs, pair-weighted over the five criteria). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 51 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Themis-RM-32B | 91.82 | |
| 2 | Themis-RM-14B | 91.19 | |
| 3 | Themis-RM-8B | 89.78 | |
| 4 | Themis-RM-4B | 88.39 | |
| 5 | Themis-RM-1.7B | 83.04 | |
| 6 | Athene-RM-70B | 81.19 | |
| 7 | Skywork-Reward-V2-Qwen3-8B | 79.97 | |
| 8 | Skywork-Reward-V2-Qwen3-4B | 79.27 | |
| 9 | Themis-RM-0.6B | 79.26 | |
| 10 | CodeScaler-8B | 79.12 |
Interactive version: theaggregate.ai/benchmark?slug=themis-coderewardbench · How It Works · Data refreshed daily, snapshot 2026-11-12.