Skip to content

For clean Markdown of any page, append .md to the page URL. For a complete documentation index, see For full documentation content, see For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at

Turn detection and interruptions

For the complete documentation index, see llms.txt

The Voice Agent API handles turn detection and interruptions automatically with intelligent interruption handling. Decisions are semantic, based on the meaning of what the user actually said, not just on silence or volume.

You don't need to wire anything in or configure it on your end. You get great out-of-the-box performance for both turn-taking and barge-in.


How it works

Traditional voice agents rely on voice activity detection (VAD) and silence thresholds to decide when a user is done talking or trying to interrupt. That works for clean speech, but it falls apart in real conversations: the agent gets cut off by a quick "uh-huh", or it fails to react when the user clearly wants it to stop.

The Voice Agent API isn't a single VAD threshold or a thin wrapper around STT, LLM, and TTS. It's an orchestrated pipeline of specialized models that work together to interpret what the user is actually doing in real time:

  • Voice focus: server-side noise cancellation that suppresses background noise and side speakers so the rest of the stack only sees the primary speaker. Enabled by default on every session.
  • Speech recognition and understanding: Universal-3 Pro converts the user's audio into transcripts, with formatting, context, and conversational cues that go beyond raw text.
  • Intelligent turn-taking: semantic detection of when the user has actually finished a thought, instead of waiting on fixed silence thresholds.
  • Intelligent interruption handling: distinguishes back-channels ("uh-huh", "yeah") from real barge-in ("wait, stop") so the agent only stops when the user genuinely wants the floor.
  • Agent reasoning and speech synthesis: the LLM and TTS that generate and stream the agent's response back to the user.

You don't see any of this directly. From your client's perspective, the server emits the right events at the right time and you react to them. The rest of this page covers the behaviors you can rely on and the events that go with them.


Semantic interruptions

While the agent is speaking, the API classifies user speech as either a back-channel or a true interruption.

Back-channeling

Short verbal acknowledgements that show the user is engaged but not trying to take the floor. The agent keeps speaking and the API does not emit an interruption.

Examples:

  • "Uh-huh"
  • "Okay"
  • "Awesome"
  • "Yeah, makes sense"
  • "Mm-hmm"

True interruptions

Phrases that signal the user wants the agent to stop. The API immediately interrupts the agent.

Examples:

  • "Wait, stop"
  • "Sorry, that's not right"
  • "Okay, wait a minute"
  • "Hold on"

When a true interruption is detected, the server emits:

  • reply.done with status: "interrupted"
  • transcript.agent with interrupted: true and text trimmed to what the user actually heard before being cut off.

See Handling interruptions for the client-side audio flush pattern.


Semantic turn detection

The API also decides when the user has finished a turn based on what they said, not just on silence. Instead of waiting for a fixed silence window, it uses the meaning of the user's speech to decide whether they're done, so the agent doesn't cut you off mid-thought, and doesn't sit on long pauses after you've clearly finished.

A typical user turn produces:

  1. input.speech.started when the user begins speaking.
  2. transcript.user.delta events with partial transcripts as the user keeps talking.
  3. input.speech.stopped when the turn is detected as ended.
  4. transcript.user with the final transcript.
  5. reply.started as the agent begins generating a response.

You don't need to send any signal to end a turn. The API handles it for you.


Configuration

Semantic turn detection and interruption handling are on by default and tuned for typical conversational use cases. For most agents, the right move is to leave them alone.

If you do need to adjust sensitivity, for example to be more patient in a noisy environment or to disable barge-in entirely, you can override the underlying VAD knobs via session.input.turn_detection:

FieldDescription
vad_thresholdSpeech detection sensitivity (0.0–1.0). Lower = more sensitive to speech.
min_silenceMinimum silence to consider a confident end-of-turn, in milliseconds.
max_silenceMaximum silence before forcing end-of-turn, in milliseconds.
interrupt_responseWhether user speech can interrupt the agent. Set false to disable barge-in.

See Session configuration → Turn detection for the full reference, default values, and example payloads.

If the agent keeps interrupting itself, the microphone is picking up the agent's own TTS output. Use headphones or switch to a browser-based client (which provides echo cancellation). See Troubleshooting for more detail.