ImmersedPrivacy - Inferred Privacy Planning (Audio as Text): leaderboard
History-conditioned privacy adherence in ImmersedPrivacy (Wang et al., 2026), Tier 3: the model watches a simulated video, with a verbatim transcript of the observed dialogue instead of its audio, in which a person hides an item and asks that it stay private, then gets a conflicting command and selects which of three candidate actions to perform. Exact match in percent (both task actions chosen, the privacy-violating one avoided); higher is better.
Metric: Exact match (%; Tier 3: share of cases where the model selects exactly the two task actions that leave alone the item an observed person wanted kept private; audio replaced by a text description of the soundscape (Tier 2) or the dialogue transcript (Tier 3)). Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) (High) | 53 | #91 (Gemini 3.1 Pro (Preview)) |
| 2 | Gemini 3.1 Pro (Preview) (Low) | 50 | #91 (Gemini 3.1 Pro (Preview)) |
| 3 | Qwen 3.5 27B (Non-reasoning) | 30 | #358 (Qwen 3.5 27B) |
| 4 | Qwen 3.5 27B (Thinking) | 18 | #358 (Qwen 3.5 27B) |
| 5 | Gemini 3 Flash (Preview) (High) | 18 | #141 (Gemini 3 Flash (Preview)) |
| 6 | Gemini 3 Flash (Preview) (Low) | 11 | #141 (Gemini 3 Flash (Preview)) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=immersedprivacy-inferred-privacy-planning-audio-as-text · How It Works · Data refreshed daily, snapshot 2026-11-12.