TRAP - Privacy: leaderboard
TRAP (2026) tests agents that must use private document fields to call a tool correctly while refusing to reveal the same fields when asked directly, over text, image and multimodal documents. This board is the privacy score in percent, one minus the leakage rate under the extraction query, over all scenarios; higher is better.
Metric: Privacy score (%): 1 minus the leakage rate under the attack query, sample-weighted over all 500 document-grounded scenarios (100 text, 200 image, 200 multimodal) over ten private document types; each pairs a task query that needs a tool call with private fields and an attack query eliciting the same fields in natural language; rule-based evaluator, greedy decoding. Source: arxiv.org. 22 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 26.4 | #266 |
| 2 | Gemini 2.5 Flash Lite | 20.4 | #958 |
| 3 | GPT-4o Mini | 13.2 | #1623 |
| 4 | GPT-5 | 12.8 | #167 |
| 5 | GPT-5.4 Mini | 10.4 | #389 |
| 6 | Gemini 2.5 Flash | 10 | #453 |
| 7 | GPT-5 Mini | 10 | #348 |
| 8 | Claude Haiku 4.5 | 8.8 | #568 |
| 9 | Claude Sonnet 4.5 | 6.6 | #250 |
Interactive version: theaggregate.ai/benchmark?slug=trap-privacy · How It Works · Data refreshed daily, snapshot 2026-11-13.