EchoChain: leaderboard

Full-duplex voice assistants on EchoChain (2026): in 200 interrupted conversations the user barges in mid-response with information that changes the task, and human annotators grade the continuation blind against an instance-specific rubric. Share of conversations whose post-interruption response satisfies every rubric criterion, in percent; higher is better.

Metric: Mean pass rate (%): share of conversations in which the post-interruption response satisfies every rubric criterion, over 200 human-screened interrupted conversations in which the user barges in mid-response with new task information (pre-generated cloned-voice audio, fixed barge-in timing), each model response graded blind by human annotators against an instance-specific rubric; higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 4 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Grok Voice Agent48.5
2GPT Realtime 2025-08-2845
3Nova 2 Sonic26.5
4Gemini Live 2.5 Flash Native Audio16.5

Interactive version: theaggregate.ai/benchmark?slug=echochain · How It Works · Data refreshed daily, snapshot 2026-11-21.