Conversational Voice AI From Robotic IVRs to Natural Conversations

"Press 1 for sales, 2 for support, 3 for billing…" We've all suffered through an IVR. Endless menus, voice recognition that fails 40% of the time, and a sense of frustration that starts before you even talk to anyone. Cadences' conversational voice AI eliminates all of that: an artificial intelligence agent answers the call, understands what you say, queries your history in real time and responds with a natural voice in whatever language you prefer.

The fundamental difference isn't just technological — it's experiential. An IVR is a fixed decision tree; a voice AI agent is an open conversation with real-time data access.

Voice AI in Cadences works as a real-time bridge between three systems. There are no polling intermediaries or processing queues — everything happens over bidirectional WebSocket with sub-500ms latency.

When Twilio receives a call, it executes a webhook that generates dynamic TwiML. This XML tells Twilio to open a WebSocket Media Stream to the Durable Object instead of playing an IVR, injecting business context as parameters:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="wss://cadences.app/api/elevenlabs/media-stream">
      <Parameter name="agentId"    value="agent_abc123" />
      <Parameter name="language"   value="en" />
      <Parameter name="callerName" value="John Smith" />
      <Parameter name="contextPrompt"
                value="Dental clinic. Patient with appointment on 02/15. Treatment: orthodontics." />
    </Stream>
  </Connect>
</Response>

Notice the contextPrompt parameter: this is where you inject the business information. The ElevenLabs agent receives this context before the first word and can use it throughout the entire conversation. "Good morning John, I see you have an orthodontics appointment on February 15th. How can I help you?"

What turns a generic voice agent into a truly useful one is context. The Durable Object sends an initialization message to ElevenLabs with the conversation config and the caller's dynamic variables:

// Message sent when connecting to ElevenLabs
const initMessage = {
  type: 'conversation_initiation_client_data',
  conversation_config_override: {
    agent: {
      language: 'en',
      prompt: {
        prompt: contextPrompt  // Business context
      },
      first_message: 'Hello John, welcome to the Dental Clinic...'
    }
  },
  dynamic_variables: {
    caller_number: '+14155551234',
    caller_name:   'John Smith',
    call_sid:      'CA1234567890abcdef',
    workflow_id:   'wf_dental_appointment'
  }
};

Dynamic variables allow the same voice agent to behave differently depending on who calls. If the CRM says John has an overdue invoice, the agent knows it before he even asks.

Cadences includes a catalog of configurable voices with fine control over stability, expressiveness and similarity. The eleven_multilingual_v2 model allows the same voice to speak Spanish, English, French or Japanese without changing profiles.

Besides incoming calls, Cadences enables outbound calls from any storefront via API. A clinic can automate appointment reminders, an e-commerce can confirm orders, or a sales team can follow up — all with personalized voice AI.

POST /api/storefront/voice-call

{
  "phoneNumber": "+14155551234",
  "cadences_prompt": "Remind John about his orthodontics appointment tomorrow at 10:00 AM. Ask if he confirms or wants to reschedule.",
  "callType": "reminder",
  "metadata": {
    "patientId": "pat_001",
    "appointmentId": "apt_2025_0215",
    "organizationId": "org_clinic23"
  }
}

// Response
{
  "success": true,
  "callId": "call_1705312800",
  "conversationId": "conv_abc123",
  "status": "initiated"
}

The proxy handles authentication with ElevenLabs and Twilio — your storefront never exposes API keys. The call is logged in D1 with the conversation ID for later follow-up.

Thanks to the eleven_multilingual_v2 model, the same voice can speak in any of these languages without needing to configure separate voices. Simply change the language parameter in the agent configuration.

Beyond telephony, Cadences includes Audio Hub: an Electron desktop application for generating, previewing and managing AI audio content. Batch Text-to-Speech, voice cloning, and export in multiple formats — all connected to the same ElevenLabs infrastructure.

Voice AI has strict controls to prevent abuse and manage costs. Each parameter is configurable per organization.

The voice AI infrastructure in Cadences runs on a pure serverless model. No servers running 24/7, no monthly infrastructure minimums. The bill is composed of three elements:

Conversational voice AI in Cadences isn't a secondary feature — it's a first-class interface on the same level as the web dashboard or the REST API. A voice agent has access to the same CRM, the same pipeline, the same data as any other entry point into the system.

The combination of Twilio (global phone network), a Durable Object as a smart bridge, and ElevenLabs (neural voice in 28+ languages) creates a voice stack that is simultaneously powerful and simple to configure. You don't need telephony engineers, you don't need to record IVR prompts, you don't need to hire a call center.