ROME Safety Judgment - Implicit Risks: leaderboard
ROME Safety Judgment Implicit Risks scores zero-shot safety judges on unsafe agent trajectories whose harmful actions are reworded as routine technical operations, balanced against safe trajectories (ROME, 2026). The score is F1 in percent for flagging the unsafe ones; higher is better.
Metric: F1 (%) of unsafe-trajectory detection on the 100 ROME rewrites of R-Judge unsafe trajectories whose harm is cloaked in neutral technical jargon; the model judges zero-shot (temperature 0) whether an LLM-agent trajectory is safe or unsafe; unsafe is the positive class; each condition pairs 100 unsafe trajectories with the same 100 safe ones; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude 3.7 Sonnet | 63.47 | #470 |
| 2 | ROME DeepSeek-R1 (checkpoint unspecified) | 57.72 | |
| 3 | Qwen 3 8B | 42.86 | #1873 |
| 4 | Qwen 3 235B A22B | 42.53 | #628 |
| 5 | GPT-4o (2024-11-20) | 31.46 | #859 |
| 6 | ROME DeepSeek-V3 (checkpoint unspecified) | 27.03 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=rome-safety-judgment-implicit-risks · How It Works · Data refreshed daily, snapshot 2026-11-21.