We have noticed a counterintuitive pattern: many voice agents ship with green network metrics, yet the user experience feels like a call-center agent who can't listen. This article breaks "full-duplex" into three layers—transport, inference, and interaction cognition—and shows that production incidents almost always occur when four state sources (capture, model response, playback queue, and dialogue history) give conflicting answers to the same event.
What this actually is
The author's core argument: WebRTC's sendrecv (bidirectional send/receive) only solves "can audio flow both ways at once," not "who should speak, when to yield, and whether the other side is barging in or backchanneling." A genuinely usable full-duplex voice agent requires three layers to be wired together simultaneously.
The transport layer handles simultaneous bidirectional media, guaranteed by protocols like WebRTC. The inference layer requires new input to be able to alter the in-progress output, rather than waiting for the current reply to finish playing before processing. The interaction cognition layer must classify a stretch of overlapping audio as backchannel ("mm-hmm," "right"), explicit takeover ("wait," "that's wrong"), a thinking filler, or a bystander/background noise. Each layer can succeed or fail independently, and the industry bundling them under a single word—"full-duplex"—is the root cause of the problem being obscured.
Even trickier: the four runtime states tend to desynchronize. The model has canceled but the speaker is still playing. Playback has stopped but the dialogue history has logged content the user never heard. The backend creates a new turn from a bystander's utterance. These splits don't crash the system; they just make conversations "look normal, sound off."
How the industry sees it
Engineering practices that support the author's view are taking shape. The OpenAI Realtime API's speech_started / speech_stopped events and interrupt_response switch are treated as "boundary samples"—they give the application the ability to build an interruption pathway, but the events and switch themselves don't answer "which type of interaction intent is this overlap." The article recommends making turn-taking decisions explicit as structured output with reason, confidence, and consequence (e.g., decision=BACKCHANNEL_ACCEPT, action=keep_speaking_and_keep_listening), rather than burying all logic in VAD (Voice Activity Detection—the module that simply decides "is someone speaking") thresholds.
But there are cautionary counter-voices worth flagging. Breaking full-duplex into three layers and mandating explicit decision objects sounds engineering-rigorous, but in practice it may regress an ability that should be learned end-to-end by the model back into a rule-based system. Some researchers worry that over-structured "turn-taking control" will strip voice agents of the fluidity of large models and drag them back to the mechanical feel of traditional IVR (phone-menu systems). In other words: should the interaction cognition layer be left to the model to learn, or written as an observable decision tree? The industry has no consensus.
What it means for regular people
For enterprise IT: when evaluating voice customer service or voice agent vendors, don't just ask "do you support WebRTC" or "what's the latency in milliseconds"—ask "how does the dialogue history get corrected after an interruption" and "will it yield the floor when the other side says 'mm-hmm'."
For working professionals: sales reps, customer service agents, and product managers who frequently meet with AI voice assistants will clearly feel, over the next year, the gap between "products that actually listen" and "products that still barge in"—and this will directly drive product selection.
For the consumer market: the "unnatural feel" of smart speakers, in-car assistants, and AI customer service won't disappear in the short term. The technical problem is now clearly defined, but the engineering cost of shipping it is high, and a genuinely fluid conversational experience may still need one or two more version iterations.