VideoFDB - Perception - TOR-Alignment: leaderboard
VideoFDB (Mazumdar et al., 2026) tests full-duplex speech agents on short conversational videos with nonverbal cues, streaming both the video and the audio. This board is TOR-Alignment, the share in percent of timing-relevant held-out test clips on which the agent takes or leaves the floor as the cue requires (stay silent, keep speaking, yield within 1.5 seconds, hand off smoothly or backchannel), computed from word-level timestamps; higher is better.
Metric: TOR-Alignment (%; share of the timing-relevant held-out perception test clips on which the agent's floor-taking matches the expected timing class: stay silent, keep speaking, yield within 1.5 s, take a smooth handoff or produce a backchannel; measured deterministically from word-level timestamps of the agent's audio, streamed through a realtime harness with both the video and the audio stream on; the median latency printed in the same cell is not read). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | MiniCPM-o 4.5 | 73 | |
| 2 | Gemini 2.5 Native Audio | 72 | |
| 3 | gpt-realtime | 72 | |
| 4 | Gemini 3.1 Flash Live | 66 | |
| 5 | gpt-realtime-mini | 66 | |
| 6 | Mini-Omni2 | 64 | |
| 7 | VITA-1.5 | 58 | #3009 |
Interactive version: theaggregate.ai/benchmark?slug=videofdb-perception-tor-alignment · How It Works · Data refreshed daily, snapshot 2026-11-13.