PolicyShiftBench - Shift Split (PSS): leaderboard
PolicyShiftBench (2026) tests policy-adaptive image guardrails: a model must block or pass an image according to the runtime policy it is given, and the same image flips labels under different policies. Policy Shift Score in percent on the held-out policy split; higher is better.
Metric: Policy Shift Score (%; over groups of the same image and risk category, the share of label-flipping policy pairs where the model is right on both the permissive and the strict policy, macro-averaged over groups; policy-conditioned image guardrailing: decide whether an image violates the supplied runtime policy bundle (one policy variant per risk category, seven categories); Shift split, 1,000 instances over 152 images under 12 held-out policy definitions; mean of three evaluations). Source: arxiv.org. Saturation forecast: Around February 2027. 19 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 55.9 | #144 |
| 2 | GPT-5.4 | 48.4 | #123 |
| 3 | Qwen 3.5 35B A3B (Thinking) | 23.3 | #482 (Qwen 3.5 35B A3B) |
| 4 | Claude Sonnet 4.6 | 11.8 | #161 |
| 5 | Qwen 3.5 35B A3B (Non-reasoning) | 11.8 | #482 (Qwen 3.5 35B A3B) |
| 6 | Qwen 3.5 4B (Thinking) | 9.3 | #1175 (Qwen 3.5 4B) |
| 7 | Qwen 3.5 4B (Non-reasoning) | 5.8 | #1175 (Qwen 3.5 4B) |
| 8 | Qwen 3.5 0.8B (Non-reasoning) | 1.1 | #8081 (Qwen 3.5 0.8B) |
| 9 | Qwen 2.5 VL 7B | 0 | #1787 |
| 10 | Qwen 3.5 2B (Non-reasoning) | 0 | #4810 (Qwen 3.5 2B) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=policyshiftbench-shift-split-pss · How It Works · Data refreshed daily, snapshot 2026-11-13.