TAF-MED - Any-Turn Unsafe Rate: leaderboard
TAF-MED (arXiv 2608.10258): 500 physician-reviewed synthetic three-turn conversations in which the user says they will treat a serious, critical or life-threatening condition themselves and keeps asking for medication guidance through fixed follow-ups. A rubric judge validated against physicians (94.3% agreement) labels each turn Safe, Leaky or Unsafe; no system prompt, temperature 0, reasoning modes off where configurable. Score: share of conversations (%) with an Unsafe response at any turn. Lower is better; over-refusal is not measured. Source: https://arxiv.org/abs/2608.10258
Metric: Any-turn unsafe rate (%; share of the 500 conversations with an Unsafe response at any of the three turns; lower is better). Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Grok 4.3 | 35.6 | #186 |
| 2 | Claude Sonnet 4.6 | 49 | #90 |
| 3 | Claude Opus 4.6 | 58.8 | #62 |
| 4 | GPT-5.4 | 74.2 | #78 |
| 5 | GPT-5.4 Mini | 80.8 | #208 |
| 6 | DeepSeek V4 Pro | 86.6 | #100 |
| 7 | Llama 4 Maverick | 90.6 | #467 |
| 8 | Gemini 2.5 Pro | 97.2 | #151 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=taf-med-any-turn-unsafe-rate · How It Works · Data refreshed daily, snapshot 2026-10-19.