RFEval - Context Understanding: leaderboard
Metric: Contrast-conditional reasoning faithfulness (%): after a counterfactual flawed reasoning segment is injected into the model's own thinking, the share of pairs in which the continued reasoning stays consistent with that stance and the stance causally carries into the final answer, computed only over the model's own contrast pairs (injection opposes its baseline stance; well-formed outputs); stances extracted by an o3 judge; greedy decoding; Context Understanding task (1,093 items from PubMedQA); a property of reasoning traces rather than task accuracy; higher is better. Source: arxiv.org. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 32B (Thinking) | 77.66 | #424 (Qwen 3 32B) |
| 2 | DeepSeek-R1-Distill-Qwen-7B | 76.31 | #1400 |
| 3 | DeepSeek R1 Distill Llama 8B | 64.63 | #1282 |
| 4 | DeepSeek R1 Distill Qwen 32B | 63.16 | #640 |
| 5 | DeepSeek R1 Distill Llama 70B | 51.4 | #582 |
| 6 | Qwen 3 8B (Thinking) | 43.97 | #667 (Qwen 3 8B) |
| 7 | GPT-OSS-20B | 33.93 | #499 |
| 8 | GPT-OSS-120B | 13.67 | #330 |
Interactive version: theaggregate.ai/benchmark?slug=rfeval-context-understanding · How It Works · Data refreshed daily, snapshot 2026-10-11.