VoxSafeBench - Audio-Conditioned Privacy: leaderboard

VoxSafeBench (2026) Tier 2 privacy test for speech language models: a user asks about a sensitive personal topic while background voices show that other people can overhear. The score is the percentage of responses that warn the user or defer instead of answering directly, judged by DeepSeek-V3, higher is better.

Metric: Privacy awareness rate (%), the second part of the DAR / PAR pair: spoken English and Chinese questions (aggregated) on sensitive topics such as pregnancy, layoffs or family conflict whose background audio shows that bystanders can hear; the share of responses that warn the user or defer instead of answering directly, labeled by DeepSeek-V3; higher is better. Source: arxiv.org. 7 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3 Pro94.3#145
2Gemini 3 Flash72.8#175
3MiMo-Audio-7B-InstructThinking53#1336
4Qwen 3 Omni 30B A3B (Thinking)50.3
5Qwen3 Omni 30B A3B Instruct14.3#930
6GPT-4o Audio8.3#1129
7Kimi-Audio4.5#1735

Interactive version: theaggregate.ai/benchmark?slug=voxsafebench-audio-conditioned-privacy · How It Works · Data refreshed daily, snapshot 2026-11-21.