IF-RewardBench - Constraint Assessment: leaderboard
Metric: Kendall tau-b (-1 to 1) between the quality scores implied by the judge's per-constraint verdicts (share of checklist constraints judged followed) and the human-verified preference graph, averaged over IF-RewardBench's three instruction types (842 instructions); higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 14 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Pro | 0.61 | #77 |
| 2 | Gemini 3 Flash | 0.57 | #93 |
| 3 | GPT-5.1 | 0.53 | #131 |
| 4 | GPT-5 Mini | 0.52 | #176 |
| 5 | GLM-4.6 (Thinking) | 0.42 | #246 (GLM-4.6) |
| 6 | DeepSeek V3.2 | 0.4 | #198 |
| 7 | QwQ-32B | 0.36 | #410 |
| 8 | GLM 4.5 Air (Thinking) | 0.31 | #377 (GLM 4.5 Air) |
| 9 | Qwen 3 32B (Thinking) | 0.28 | #424 (Qwen 3 32B) |
| 10 | Llama 3.3 70B Instruct | 0.24 | #520 |
| 11 | Qwen 3 8B (Thinking) | 0.23 | #667 (Qwen 3 8B) |
| 12 | Qwen 2.5 72B Instruct | 0.18 | #436 |
| 13 | Qwen 2.5 7B Instruct | 0.09 | #846 |
| 14 | Llama 3.1 8B Instruct | 0.09 | #1018 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=if-rewardbench-constraint-assessment · How It Works · Data refreshed daily, snapshot 2026-10-11.