DPrivBench - Permute-and-Flip: leaderboard

Differential privacy verification for the report-noisy-max with exponential noise (permute-and-flip) on the mechanism-level instances of DPrivBench (2026), where the model decides whether a stated differential privacy guarantee holds for a textbook mechanism instantiated with a concrete query, no tools, five seeds. Accuracy in percent; higher is better.

Metric: Accuracy (0-1, times 100) on the report-noisy-max with exponential noise (permute-and-flip) instances of the DPrivBench Category 1 mechanism-level instances (a mechanism template instantiated with a concrete query function from a 49-function bank; positive instances calibrate the noise to the true sensitivity, paired negatives under-calibrate it); binary yes/no verdict on whether the stated differential privacy guarantee holds, five seeds, no tools; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5High99.6#178
2Gemini 3 Pro93.9#145
3Gemini 2.5 Flash66.5#463
4Claude Opus 4.565.7#160
5DeepSeek V3.1Non-reasoning65.1#511
6Claude Sonnet 4.560#261
7DeepSeek R152.2#482
8Qwen 3 30B A3B 2507 Instruct52#1100
9Goedel-Prover-V2-32B50.6
10Qwen 3 30B A3B 2507 (Thinking)46.5#764

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=dprivbench-permute-and-flip · How It Works · Data refreshed daily, snapshot 2026-11-21.