Components
System inventory
- Twilio webhook
- Media WebSocket
- STT
- Turn manager
- LLM
- TTS
- Call trace
Cascaded telephony voice agent
A phone call streams through telephony, speech recognition, an orchestrated LLM, and streaming speech synthesis.
Runtime flow
What happens, in order.
The flow becomes the backbone of the audit trace. Every important boundary needs an input, an output, timing, and an owner.
- 01
Twilio sends a signed POST to /api/webhooks/twilio/voice
- 02
The server validates the request, creates a session, and returns <Connect><Stream> TwiML
- 03
Twilio opens a bidirectional WSS stream carrying base64 μ-law 8 kHz mono audio
- 04
The voice gateway forwards audio to streaming STT and combines finalized segments until end-of-turn
- 05
The orchestrator runs the system prompt, RAG, tools, policy, and conversation state
- 06
Streaming model text is synthesized as μ-law 8 kHz audio and sent to Twilio media messages
- 07
Barge-in cancels stale model/TTS work and sends clear before the next turn
- 08
On stop/status callbacks, the system persists transcript, timing, tool, handoff, and outcome evidence
Test packs
Required coverage
- Webhook contract
- Media format
- STT entities
- Latency
- Barge-in
- TTS
- Handoff
Evidence
Capture contract
- Call SID
- Stream events
- Audio fixtures
- Transcript
- Turn timings
- Tool calls
- End reason
Voice implementation boundary
The evaluator models the complete call without becoming the customer’s phone platform.
The HTTP webhook, media WebSocket, STT, orchestration, TTS, and status callback are separate observable boundaries. A connected system can provide live traces; the Northstar reference target can provide controlled fixtures for regression tests.
POST /api/webhooks/twilio/voicereturns signed-call TwiMLWSS /media-streamreceives and sends μ-law 8 kHz media eventsPOST /api/webhooks/twilio/call-statusfinalizes the trace and outcome