NPC-RubricBench Focused - Error Rate: leaderboard
NPC-RubricBench v1 evaluates patient-specific nasopharyngeal care in Chinese. The focused formulation contains 941 patient-phase units from 405 patients and evaluates 30 model configurations, with a maximum of 8192 output tokens. Local models use temperature 0 and topP 1, with explicit thinking/instruct (T/I) variants. GPT 5.1 uses the 2025-11-13 snapshot, and GPT 5.6 Luna is also evaluated; both use high reasoning effort. Sonnet 4.6 uses temperature 1; Gemini 3.1 Pro uses temperature 0 with thought; DeepSeek V4 Pro uses temperature 1 with high thinking, and Flash uses temperature 1. Judge criteria cover decision quality (DQ), grounding (EG), fact faithfulness (EF), specificity (SD), and alignment (SA). The error factor is -0.5, and scores are normalized to 0–100. Retained reasoning is scored, and source-qualified run identities remain distinct. The study does not establish access to private patient records. Phase populations are 191 workup, 179 diagnosis, 130 staging, 177 treatment, 103 toxicity, and 161 followup units. The violation percentage counts responses with at least one EF or SA criterion not met. Error and omission rates use the preset applicable-rubric denominator, not the response count. Internal and external cohorts are reported separately. Metric: Rate (%).
Metric: Rate (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 30 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | NPC-RubricBench GPT-5.1 native configuration | 3.71 | |
| 2 | NPC-RubricBench GPT-5.6-Luna native configuration | 3.78 | |
| 3 | NPC-RubricBench Gemini 3.1 native configuration | 4.12 | |
| 4 | NPC-RubricBench DeepSeek-V4-Pro-T native configuration | 4.62 | |
| 5 | NPC-RubricBench Qwen3.6-27B-T native configuration | 4.64 | |
| 6 | NPC-RubricBench Qwen3.5-35B-A3B-T native configuration | 5.1 | |
| 7 | NPC-RubricBench Baichuan-M2-32B native configuration | 5.59 | |
| 8 | NPC-RubricBench Qwen3-30B-A3B-T native configuration | 5.78 | |
| 9 | NPC-RubricBench Qwen3-VL-30B-A3B-T native configuration | 6 | |
| 10 | NPC-RubricBench GPT-OSS-20B native configuration | 6.09 |
Interactive version: theaggregate.ai/benchmark?slug=npc-rubricbench-focused-error-rate · How It Works · Data refreshed daily, snapshot 2026-10-28.