Browser mic as a stand-in for a phone voice line: Web Speech API loop that does not transcribe its own TTS
Testing fallback for a phone voice agent (Photon + ElevenLabs) without any telephony, built 2026-11-14 in React 19 + Vite 8 + TS 7. Use window.SpeechRecognition ?? window.webkitSpeechRecognition (works in Chrome/Edge; absent in Firefox, so also give a text box). TS lib.dom may lack these types, so declare a minimal interface yourself. Settings: continuous=true and interimResults=true. Post only results with isFinal=true to the backend, and show the interim text in the UI. Chrome ends recognition after silence, so restart in onend while the call is live. To speak the backend reply (speechSynthesis + SpeechSynthesisUtterance) without the mic transcribing it, set a speaking flag, call rec.abort() before speak(), and start again in the utterance onend/onerror. The onend restart must check the flag, or recognition restarts mid-speech. Mic access needs localhost or HTTPS. Behind Caddy, build the WebSocket URL from location.protocol (wss when on https) and location.host so dev (Vite proxy) and prod are both same-origin. Not checked: Safari behaviour, and whether the abort/restart cycle drops a word at turn boundaries.
Safari follow-up (2026-11-14): the auto-restart loop works for a while on desktop Safari, then fails with error 'not-allowed'. Safari only allows recognition.start() from a user gesture, so restarts from onend or after TTS get refused; reports also call its continuous mode unreliable. What we shipped: on Safari, push-to-talk, where each tap on a Talk button builds a fresh recognizer with continuous=false for one phrase, and the tap also cancels speechSynthesis (barge-in). On Chrome/Edge, keep one continuous session open during TTS and ignore its results until 400 ms after speech ends, instead of abort/start around every utterance. Never auto-restart on not-allowed/service-not-allowed/audio-capture; show a Resume button, because the click is a gesture. Back off when sessions end within 1 s, five times in a row. Not yet re-tested on Safari by the user.