PolicyShiftBench: leaderboard
PolicyShiftBench (2026) tests policy-adaptive image guardrails: a model must block or pass an image according to the runtime policy it is given, and the same image flips labels under different policies. F1 of the block decision in percent, averaged over the training-family and held-out policy splits; higher is better.
Metric: F1 of the block decision (%; policy-conditioned image guardrailing: decide whether an image violates the supplied runtime policy bundle (one policy variant per risk category, seven categories); mean of the Adaptive split (policy families seen in training) and the Shift split (held-out policies), 2,000 instances; mean of three evaluations). Source: arxiv.org. Saturation forecast: Around February 2027. 19 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 70.6 | #144 |
| 2 | GPT-5.4 | 57.8 | #123 |
| 3 | Qwen 3.5 35B A3B (Thinking) | 52.4 | #482 (Qwen 3.5 35B A3B) |
| 4 | Qwen 3.5 35B A3B (Non-reasoning) | 48.1 | #482 (Qwen 3.5 35B A3B) |
| 5 | Qwen 3.5 4B (Non-reasoning) | 47.4 | #1175 (Qwen 3.5 4B) |
| 6 | Claude Sonnet 4.6 | 39.8 | #161 |
| 7 | Qwen 3.5 4B (Thinking) | 31.6 | #1175 (Qwen 3.5 4B) |
| 8 | Qwen 2.5 VL 7B | 20.6 | #1787 |
| 9 | Qwen 3.5 2B (Non-reasoning) | 9.1 | #4810 (Qwen 3.5 2B) |
| 10 | Qwen 3.5 0.8B (Non-reasoning) | 3.9 | #8081 (Qwen 3.5 0.8B) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=policyshiftbench · How It Works · Data refreshed daily, snapshot 2026-11-13.