When Context Flips, Safety Breaks: leaderboard
Safety benchmark scores provide incomplete evidence of deployment readiness: aligned language models often adhere to rigid rules even when a situational update flips which action is safe. We term this failure brittle safety. To diagnose it, we introduce context-flip evaluation, testing 12 models across a safety benchmark (PacifAIst) and two commonsense controls using paired variants where the nomi
Metric: PacifAIst BSR (self-reported). Source: benchmarklist.com. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3.1 8B | 77.6 | #1151 |
| 2 | Claude Sonnet 4.6 | 43.8 | #90 |
| 3 | Llama 3.3 70B | 38 | #579 |
| 4 | DeepSeek V3.1 | 35.9 | #264 |
| 5 | Mistral Small 3.1 | 35.9 | #610 |
| 6 | phi-4-14B | 35.9 | #733 |
| 7 | Gemma 3 27B | 27.8 | #607 |
| 8 | GPT-5.4 | 22.8 | #79 |
| 9 | Grok 4.20 | 18.2 | #175 |
| 10 | Gemini 3.1 Pro (Preview) | 17.2 | #58 |
| 11 | Qwen 3 32B | 16.2 | #443 |
Interactive version: theaggregate.ai/benchmark?slug=when-context-flips-safety-breaks · How It Works · Data refreshed daily, snapshot 2026-10-20.