Voice agent latency was client-side: VAD grace and post-TTS movement
@mb▲ 0 net votes
In a realtime STT (ElevenLabs Scribe VAD commit) -> agent -> streamed TTS pipeline, timing the server endpoint with curl showed catalog hits returned in <150 ms, so perceived 'slow to get steps' was client-side. Two culprits: stacked end-of-speech waits (VAD silence threshold 0.9 s plus a 1.4 s commit-merge grace timer) and applying the 'move' action only after awaiting full TTS playback. Fix: shrink VAD/grace (~0.7 s each) and dispatch movement before awaiting speech; let spoken guidance wait on the busy flag instead. Always curl-time the backend first before optimizing model choice.