Themis-CodeRewardBench - Execution Efficiency: leaderboard
The execution-efficiency split of Themis-CodeRewardBench (Paul, Glavaš and Gurevych, 2026): 1,309 code preference pairs from runtime-improving commits, Pie4Perf, ECCO and EvalPerf, where the preferred response is the faster one. Preference accuracy of pointwise reward-model scores in percent; higher is better.
Metric: Preference accuracy (%; share of the benchmark's preference pairs in which the reward model scores the preferred code response above the rejected one, pointwise and reference-free; Execution Efficiency criterion, 1,309 pairs: runtime-improving commits, Pie4Perf, ECCO and EvalPerf). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 51 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Themis-RM-14B | 86.1 | |
| 2 | Themis-RM-32B | 84.95 | |
| 3 | Themis-RM-8B | 83.65 | |
| 4 | Themis-RM-4B | 83.65 | |
| 5 | Themis-RM-1.7B | 81.89 | |
| 6 | Themis-RM-0.6B | 79.98 | |
| 7 | Athene-RM-70B | 67.07 | |
| 8 | Qwen3-Nemotron-32B-GenRM-Principle | 66.77 | |
| 9 | LDL-Reward-Gemma-2-27B-v0.1 | 65.47 | |
| 10 | Llama-3.1-Tulu-3-70B-SFT-RM-RB2 | 65.16 |
Interactive version: theaggregate.ai/benchmark?slug=themis-coderewardbench-execution-efficiency · How It Works · Data refreshed daily, snapshot 2026-11-12.