Voice AI

Voice AI Employees handle real-time phone conversations -- answering inbound calls, conducting outbound campaigns, and replacing static IVR trees with natural language interactions. Instead of forcing callers through a menu of button presses, a Voice AI Employee listens, understands intent, retrieves relevant information, and responds conversationally.

This page covers the core patterns for building voice-based AI Employees on Ema's platform, from simple inbound call handling to complex multi-step voice workflows with tool calling and escalation.

What You'll Need

• Builder or Admin access to your Ema workspace.
• A provisioned phone number with SIP trunk configuration for inbound or outbound calls.
• Knowledge base content -- the same documentation, FAQs, and policy documents you would use for a chat AI Employee.
• Identity and purpose instructions -- a description of who the Voice AI Employee is, what it does, and how it should behave on calls (tone, pacing, escalation rules).
• Familiarity with the AI Employee builder -- voice workflows use the same GWE canvas as chat workflows, with a voice-specific trigger.

Why Voice?

Many customer interactions still happen over the phone -- especially in healthcare, insurance, financial services, and customer support. Traditional IVR systems are rigid: they force callers into predefined paths and cannot handle nuance. Voice AI Employees combine the natural language understanding of a chat AI Employee with the immediacy and accessibility of a phone call.

Pattern 1: Inbound Call Handling

Problem

Callers reach a phone line and expect immediate help. Traditional IVR menus are frustrating and slow. Human agents are expensive and have limited availability. The result is long hold times, high abandonment rates, and inconsistent service quality.

Components

Voice Trigger (inbound call)
 --> Speech-to-Text (transcription)
 --> Intent Classification
 --> Search knowledge base
 --> Generate response
 --> Text-to-Speech (spoken response)
 --> [If unresolved] Escalate to human agent

• Voice Trigger -- Activates the workflow when a call arrives on the provisioned phone number.
• Speech-to-Text -- Converts the caller's speech into text using high-accuracy transcription (GPT-5.4 for diarization by default).
• Workflow execution -- The transcribed text flows through the same agent pipeline as a chat workflow: intent classification, knowledge retrieval, response generation.
• Text-to-Speech -- Converts the generated text response back into natural-sounding speech for the caller.

Setup Steps

1. Create a Voice AI Employee. Navigate to AI Employees, click + New Employee, and select Voice as the interaction type.
2. Configure the phone number. Provision a phone number and configure the SIP trunk in the Voice AI Employee settings.
3. Set identity and purpose. Write clear instructions that define the AI Employee's persona for phone interactions: name, role, tone of voice, pacing, and what topics it should and should not discuss.
4. Add your knowledge base. Upload or connect the same documentation, FAQs, and policy files you use for chat-based support.
5. Build the workflow. Use the GWE canvas to construct the call handling flow. A simple pattern: Voice Trigger --> Search/v2 --> Call LLM (respond) --> Workflow Output.
6. Configure escalation. Add a branch that transfers the call to a human agent when the AI Employee cannot resolve the issue or when the caller explicitly requests a human.
7. Test with real calls. Place test calls covering your most common inquiry types. Listen to the full interaction and verify transcription accuracy, response relevance, and conversational flow.

Watch Out For

• Latency is critical. Unlike chat, voice interactions feel broken when there is more than a second or two of silence. Keep workflows short -- minimize the number of agent hops between the voice trigger and the response.
• Transcription accuracy varies. Background noise, accents, and poor audio quality all affect speech-to-text accuracy. Test with realistic audio conditions, not just quiet studio recordings.
• Conversation pacing differs from chat. Voice responses should be shorter and more conversational than chat responses. Instruct the response agent to keep answers concise and to pause for the caller's acknowledgment before continuing with lengthy explanations.
• Always provide an escalation path. Callers who cannot get help from the AI Employee and have no way to reach a human will have a very negative experience. Ensure every voice workflow includes a clear handoff mechanism.

Pattern 2: Voice with Tool Calling

Problem

Callers often need the AI Employee to take action on their behalf -- check an order status, schedule an appointment, reset a password, or file a claim. A voice AI Employee that can only answer questions but cannot act is limited in value.

Components

Voice Trigger (inbound call)
 --> Speech-to-Text
 --> Intent Classification
 --> [information] --> Search KB --> Respond
 --> [action] --> External Tool Caller --> Respond with Tool Result
 --> Text-to-Speech

This pattern extends Pattern 1 by adding an action branch. When the intent classifier identifies an action request, the workflow routes to the External Tool Caller agent, which executes the action in the connected system and returns the result to the caller via text-to-speech.

Setup Steps

1. Start from Pattern 1. Build the inbound call handling workflow first.
2. Add the intent classifier. Configure categories for information requests (handled by search + respond) and action requests (handled by tool calling).
3. Connect enterprise applications. On the Integrations page, connect the systems the AI Employee will interact with (CRM, ticketing, scheduling, etc.).
4. Enable tools for the ETC agent. Select which actions the AI Employee can perform on each connected application.
5. Enable HITL for sensitive actions. For actions that modify data (creating tickets, scheduling appointments), enable Human-in-the-Loop confirmation so the AI Employee verbally confirms the action with the caller before executing.
6. Test end-to-end. Call in, request an action, confirm the action is executed in the target system, and verify the caller receives a clear confirmation.

Watch Out For

• Confirm before acting. In a voice context, misunderstood speech can lead to the wrong action. Always confirm action parameters with the caller before executing.
• Keep HITL prompts natural. The confirmation message should sound like a natural conversation, not a form read-back. "Just to confirm, you'd like me to schedule that appointment for Thursday at 2 PM -- is that right?" is better than "Confirm: action=schedule, date=Thursday, time=14:00."
• Handle tool failures gracefully. If the external system is down, the AI Employee should say something helpful ("I'm unable to check that right now, but I can transfer you to someone who can help") rather than going silent or returning a technical error.

Design Considerations

Voice vs. Chat

Voice AI Employees and chat AI Employees share the same underlying workflow engine, but they have different user experience constraints:

Optimizing for Voice Latency

• Use the Fastest EmaFusion™ optimization mode for voice workflows to minimize model response time.
• Keep workflows linear where possible -- parallel branches add coordination overhead.
• Instruct response agents to produce short, direct answers. Avoid agents that generate lengthy explanations.

Related Resources

• Core Concepts: AI Employees -- Interaction types including voice.
• Builder Guides: Creating Your First Workflow -- The GWE canvas works the same for voice and chat.
• Agent Reference: Categorize Conversations and Route, Agentic Search and Respond
• Integrations: Integration List -- Supported external systems for tool calling.
• EmaFusion: Setup & Configuration -- Optimization modes for voice latency.