SciHazard (Deep Research Agents) - Oversafety: leaderboard
SciHazard (Shanghai AI Laboratory, 2026) measures scientific misuse risk with 2,400 hazardous questions grounded in regulated substances and documented misuse scenarios across 12 disciplines, plus 600 benign questions that reuse the same trigger words. Over-refusal is the share of the 600 benign defence, detection and emergency-response questions that are refused, in percent; lower is better; this board covers deep-research agents and agent frameworks with web retrieval.
Metric: Over-refusal rate (%; share of 600 benign defence, detection and emergency-response questions that reuse hazardous trigger words and are refused; lower is better; deep-research agents and agent frameworks with web retrieval at provider defaults). Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Sonar Pro | 5 | #722 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=scihazard-deep-research-agents-oversafety · How It Works · Data refreshed daily, snapshot 2026-11-13.