VoxSafeBench - Audio-Conditioned Privacy: leaderboard
VoxSafeBench (2026) Tier 2 privacy test for speech language models: a user asks about a sensitive personal topic while background voices show that other people can overhear. The score is the percentage of responses that warn the user or defer instead of answering directly, judged by DeepSeek-V3, higher is better.
Metric: Privacy awareness rate (%), the second part of the DAR / PAR pair: spoken English and Chinese questions (aggregated) on sensitive topics such as pregnancy, layoffs or family conflict whose background audio shows that bystanders can hear; the share of responses that warn the user or defer instead of answering directly, labeled by DeepSeek-V3; higher is better. Source: arxiv.org. 7 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3 Pro | 94.3 | #145 | |
| 2 | Gemini 3 Flash | 72.8 | #175 | |
| 3 | MiMo-Audio-7B-Instruct | Thinking | 53 | #1336 |
| 4 | Qwen 3 Omni 30B A3B (Thinking) | 50.3 | ||
| 5 | Qwen3 Omni 30B A3B Instruct | 14.3 | #930 | |
| 6 | GPT-4o Audio | 8.3 | #1129 | |
| 7 | Kimi-Audio | 4.5 | #1735 |
Interactive version: theaggregate.ai/benchmark?slug=voxsafebench-audio-conditioned-privacy · How It Works · Data refreshed daily, snapshot 2026-11-21.