SSP-Bench - Privacy: leaderboard
The privacy service of SSP-Bench (Deniz, Boshmaf and Khalil, QCRI, 2026), a dynamic benchmark that generates items from external sources and selects them for difficulty and separability. Each model gets 5,000 prompts asking it to paraphrase, summarise, redact, query or reason over legal and medical documents filled with synthetic personal data. Safe rate in percent, the share of prompts whose response contains none of the targeted personal-data spans verbatim; higher is better.
Metric: Safe rate (%; share of 5,000 PII-extraction prompts over legal and medical documents filled with synthetic PII, six attack framings, on which no targeted PII span appears verbatim in the response; SSP-Bench dynamic benchmark instance scored on the 24-model testing panel, held out from item generation and selection; proprietary models through OpenRouter, March-April 2026). Source: arxiv.org. Saturation forecast: Not forecast. 24 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Haiku 4.5 | 53.9 | #568 |
| 2 | GPT-5 Mini | 52.5 | #349 |
| 3 | Claude 3.5 Haiku | 52 | #1563 |
| 4 | GPT-4o | 47 | #741 |
| 5 | Grok 4 Fast | 45.5 | #438 |
| 6 | Gemma 4 31B | 45.3 | #393 |
| 7 | Phi-4 | 43.8 | #1920 |
| 8 | Gemma 4 26B A4B | 41.4 | #617 |
| 9 | Gemini 3 Flash (Preview) | 40.9 | #141 |
| 10 | Grok 4.1 Fast | 40.2 | #426 |
| 11 | Gemma 3 12B | 37.5 | #2013 |
| 12 | Llama 3.3 70B | 35.3 | #1511 |
| 13 | Grok 3 Mini | 34.5 | #535 |
| 14 | GPT-OSS-120B | 33.1 | #699 |
| 15 | GPT-OSS-20B | 33.1 | #1150 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=ssp-bench-privacy · How It Works · Data refreshed daily, snapshot 2026-11-12.