FinSafetyBench (PAIR) - Financial Crimes (English): leaderboard
FinSafetyBench (Hou et al., 2026) has 1,881 harmful financial instructions in aligned English and Chinese versions, built from Chinese court judgments on financial crimes, CFA ethics cases and financial items of existing red-teaming sets. This board attacks the English instructions of the 11 financial-crime categories (market manipulation, fraud, money laundering and others) with the PAIR jailbreak, in which an attacker LLM iteratively rewrites the request from the target's replies. Attack success rate in percent, counting a response only when a Qwen3-32B judge rates it a complete violation (10 of 10); lower is better.
Metric: Attack success rate (%; PAIR jailbreak; share of the benchmark's financial-crime instructions in English whose jailbroken response a Qwen3-32B judge rates a complete violation (10 of 10)). Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3 8B Instruct | 34.79 | #4652 |
| 2 | glm-4-9B (0414) | 90.52 | #2196 |
| 3 | InternLM3-8B-Instruct | 90.74 | #2586 |
| 4 | Qwen 2.5 32B Instruct | 92.12 | #1216 |
| 5 | Qwen 2.5 72B Instruct | 93.89 | #1093 |
| 6 | Mistral Small 3 | 94.53 | #1786 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=finsafetybench-pair-financial-crimes-english · How It Works · Data refreshed daily, snapshot 2026-11-12.