FinSafetyBench (FlipAttack) - Financial Crimes (English): leaderboard
FinSafetyBench (Hou et al., 2026) has 1,881 harmful financial instructions in aligned English and Chinese versions, built from Chinese court judgments on financial crimes, CFA ethics cases and financial items of existing red-teaming sets. This board attacks the English instructions of the 11 financial-crime categories (market manipulation, fraud, money laundering and others) with the FlipAttack jailbreak, which flips the characters of the request and asks the model to restore and follow it. Attack success rate in percent, counting a response only when a Qwen3-32B judge rates it a complete violation (10 of 10); lower is better.
Metric: Attack success rate (%; FlipAttack jailbreak; share of the benchmark's financial-crime instructions in English whose jailbroken response a Qwen3-32B judge rates a complete violation (10 of 10)). Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3 8B Instruct | 29.16 | #4652 |
| 2 | glm-4-9B (0414) | 34.51 | #2196 |
| 3 | InternLM3-8B-Instruct | 40.88 | #2586 |
| 4 | Qwen 2.5 32B Instruct | 93.06 | #1216 |
| 5 | Mistral Small 3 | 93.32 | #1786 |
| 6 | Qwen 2.5 72B Instruct | 94.26 | #1093 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=finsafetybench-flipattack-financial-crimes-english · How It Works · Data refreshed daily, snapshot 2026-11-12.