MedES - Ethical Reasoning: leaderboard
Open-ended Chinese medical-ethics set of MedES (Jin et al., Shanghai Jiao Tong University, AAAI 2026): 4,004 realistic user queries in 12 high-risk clinical scenarios such as organ transplantation and assisted reproduction, generated from rules in Chinese laws, standards and guidelines. The authors' fine-tuned QwQ-32B evaluator gives -1 to a response that states wrong or unlawful content or encourages unsafe behavior, and otherwise the mean of four binary quality checks (issue identified, norms cited, actionable advice, empathy); the score is the mean on -1 to 1, higher is better.
Metric: Comprehensive score (-1 to 1; 4,004 open-ended medical-ethics queries: -1 per risky response, otherwise the mean of four binary quality checks, fine-tuned QwQ-32B evaluator). Source: arxiv.org. Saturation forecast: Estimated already saturated. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek R1 | 0.81 | #467 |
| 2 | DeepSeek V3 | 0.76 | #705 |
| 3 | GPT-4 Turbo | 0.44 | #1002 |
| 4 | GPT-4 | 0.34 | #996 |
| 5 | DeepSeek-R1-Distill-Qwen-7B | 0.23 | #8045 |
| 6 | GPT-3.5 | 0.22 | #2430 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=medes-ethical-reasoning · How It Works · Data refreshed daily, snapshot 2026-11-12.