VideoFDB - Perception (Audio Only): leaderboard

Full-duplex conversation from VideoFDB (Mazumdar et al., 2026): 105 held-out test clips of seven nonverbal conversational dynamics, such as pauses, backchannels, laughter and interruptions, are streamed in real time to a full-duplex speech agent with the video stream disabled, so it hears but does not see the user. A gpt-4o judge scores the transcribed, time-stamped response from 0 to 5 on fluency, conversational flow and semantic grounding, and this board averages the three; higher is better. This is the audio-only (A2A) comparison run of the paper.

Metric: gpt-4o judge score (0-5; the 105 held-out test clips of the seven nonverbal conversational dynamics scored for perception (237 clips of 11 dynamics in the full benchmark), streamed in real time; the judge reads the transcribed, time-stamped agent response with the clip annotations; mean of the fluency, conversational-flow and semantic-grounding rubrics; the same clips with the video stream disabled). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 6 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=videofdb-perception-audio-only · How It Works · Data refreshed daily, snapshot 2026-11-12.