FinSafetyBench (FlipAttack) - Ethical Violations (English): leaderboard
FinSafetyBench (Hou et al., 2026) has 1,881 harmful financial instructions in aligned English and Chinese versions, built from Chinese court judgments on financial crimes, CFA ethics cases and financial items of existing red-teaming sets. This board attacks the English instructions of the three financial ethical-violation categories (harming customer or employer interests, lack of independence) with the FlipAttack jailbreak, which flips the characters of the request and asks the model to restore and follow it. Attack success rate in percent, counting a response only when a Qwen3-32B judge rates it a complete violation (10 of 10); lower is better.
Metric: Attack success rate (%; FlipAttack jailbreak; share of the benchmark's ethical-violation instructions in English whose jailbroken response a Qwen3-32B judge rates a complete violation (10 of 10)). Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | InternLM3-8B-Instruct | 18.17 | #2586 |
| 2 | glm-4-9B (0414) | 20.02 | #2196 |
| 3 | Llama 3 8B Instruct | 27.99 | #4652 |
| 4 | Mistral Small 3 | 68.97 | #1786 |
| 5 | Qwen 2.5 32B Instruct | 71.34 | #1216 |
| 6 | Qwen 2.5 72B Instruct | 74.56 | #1093 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=finsafetybench-flipattack-ethical-violations-english · How It Works · Data refreshed daily, snapshot 2026-11-12.