Building a Voice AI Employee
A Voice AI Employee answers phone calls and talks with callers in real time. It listens, speaks, looks things up, takes actions in your systems, and hands off to a person when a call needs one. A caller reaches it by dialing a phone number, and you can also place a test call from your browser while you build.
This guide is the complete builder walkthrough: why Ema Voice is built the way it is, how a call is structured, the setup wizard, how training builds your call flow from real calls, the advanced settings, Call Stages (the building block for complex flows), and the audit and metrics you use to watch it in production.
Beta. Voice AI Employees are in beta and rolling out gradually. If you don't see voice options when you create an AI Employee, it isn't enabled for your Workspace yet, so contact your administrator.
Inbound calls only. A Voice AI Employee answers calls that come to it: a caller dials your number and the AI Employee picks up. Placing outbound calls is not supported today. We're working to expand the scope in future releases.
Why Ema Voice
Ema Voice is built to feel like a person, act on your real systems, and stay accountable for every call.
- Realtime and cascaded speech pipelines. Each Call Stage can run a single realtime speech-to-speech model for the lowest latency, or a speech-to-text then LLM then text-to-speech pipeline when you want more control and inspectability. You choose per stage, in the same call.
- Runs complex work while it talks. During a call it can trigger other AI Employees in the background to run multi-step workflows (look up an account, file a claim, update a record) and bring the result back into the conversation.
- Built from your past calls. An automated training pipeline reads your real call transcripts and standard operating procedures, builds the call flow, then runs automated test calls to score and refine it before you go live.
- Talks at a human pace, and you can interrupt it. Callers can talk over the AI Employee and it stops to listen. If a caller changes their mind mid-sentence ("actually, ticket 4471, not 4417"), the correction takes, and it doesn't read back the answer the caller already moved past.
- Every call is auditable. For each call you can review the redacted transcript synced to the recording, the fields that were extracted, when the AI Employee transferred or ended the call, and leave a pass/fail review with comments.
How a Voice AI Employee works
A Voice AI Employee is a normal AI Employee with its interaction type set to Voice. It runs on a different framework from the workflows behind chat and dashboard AI Employees, and that difference is worth understanding before you build.
A call is a flow of Call Stages
Each Call Stage is one phase of the conversation, with its own Instructions, its own tools, and its own voice and speech model. While a call is in a stage, the AI Employee works on that stage's job until the conversation moves on. The simplest Voice AI Employee is a single Call Stage that handles the whole call; a complex one routes between many.
Your workflows run as tools, mid-conversation
Call Stages don't replace your workflows. At any point in a call, the AI Employee can trigger a workflow (by handing work to another AI Employee) as a tool call. It runs in the background while the AI Employee keeps talking to the caller, and the result comes back into the conversation. This is how a call does real work, like looking up an account, filing a claim, or updating a record, without leaving the caller in silence.
Why voice is built this way
Latency is the constraint that matters most on a phone call. Callers notice a pause of even half a second, so every conversational turn has a very small budget. A workflow runs a fixed graph of Agents from start to finish, which is the right model for chat and batch work, but on a call it adds delay to every turn and forces the conversation down a predetermined path. Call Stages let the AI Employee answer immediately and reach for a full workflow only at the moments that actually need one.
Setting up your Voice AI Employee
In the Create AI Employee dialog, give the AI Employee a Name and Description, set Interaction Type to Voice, and optionally pick a Group and an icon. Choose Create.
Ema provisions the voice configuration automatically and opens a setup wizard that walks you through four steps: Basic settings, Connections, Telephony settings, and Training data. You can move back to an earlier step at any time; only the first step has a required field, and your progress is saved as you go.
Step 1: Basic settings
This step defines who the AI Employee is and how it speaks.
| Setting | What it does |
|---|---|
| Goals and tasks (required) | What the AI Employee is and what it is responsible for. This is the core of its Instructions. Example: "You are a front-office support representative at Acme Corp. Help callers with order shipment, returns, and delivery timelines." This is the one field you must fill in to continue. |
| Persona and guardrails | How it should sound and the rules it must follow. Example: "Be warm, concise, and professional. Verify the caller's identity before sharing personal details, and escalate to a human when unsure." |
| Welcome message | The greeting a caller hears the moment the call connects, before the AI Employee says anything else. It comes pre-filled with a friendly default that you can edit. |
| Language | The languages the AI Employee answers in, with one marked as the default. It answers in the default language and switches only among the languages you selected. Choose from the supported list (English, Spanish, French, German, Portuguese (Brazil), Italian, Dutch, Japanese, Mandarin Chinese, Korean, Hindi, and more). |
Write voice copy the way a person would say it out loud. Sentences that read fine on screen often sound stilted when spoken. Keep the welcome message short.
Step 2: Connections
Connect what the AI Employee can draw on during a call. All three are optional and can be added later.
- Connected AI Employees: other AI Employees this one can hand work to. They run their specialized workflows behind the scenes and return a result to the call.
- Tools: actions the AI Employee can take on the caller's behalf, drawn from your Workspace's connected integrations and MCP servers. See the Integrations Hub.
- Knowledge Bases: the documents the AI Employee can search to answer caller questions, like policies, product details, or an FAQ. See Setting up a Knowledge Base.
Step 3: Telephony settings
This step connects real callers to the AI Employee and defines where it can transfer them.
Inbound number
The phone number callers dial. Add a line with Add a phone line, which offers three paths:
| Path | When to use it | What you provide |
|---|---|---|
| Import existing | You already own numbers in Twilio. | Connect your Twilio account once (Account SID and Auth Token, stored encrypted and never shown again), then pick a number to import and assign. |
| Purchase new | You want a new number. | Search by country and area code and buy one through your connected Twilio account. Purchasing incurs Twilio charges on that account. |
| Your own carrier (SIP) | You want to keep your existing carrier. | Enter a number you already own in E.164 format. Ema shows you a SIP endpoint to point your carrier at. |
Everything behind the number (the SIP trunk and call routing) is provisioned for you. A number shows as Provisioning while it is being set up and Active once it can take calls.
Agent handoff
The people and phone numbers the AI Employee can transfer a live call to. For each destination you add a Label (for example "Billing team"), a Destination (a tel: phone number or a sip: address), and When to hand off, a short description the AI Employee reads to decide when this destination applies. The AI Employee picks the right destination during the call based on those descriptions.
The When to hand off text is the single most important field on a destination. It is how the AI Employee knows to route "I want to talk to billing" to the billing team and not to sales.
Step 4: Training data
Add call transcripts and reference documents, and Ema builds the call flow from them. You can also skip this step and add training data later. See Training your Voice AI Employee for how this works.
Finish the wizard with Train AI Employee (if you added data) or Skip for now. Either way, the AI Employee is created.
Before you can go live, add at least one phone number. Publishing a Voice AI Employee requires an inbound number so callers can reach it. You can build, train, and test with browser calls before then.
Training your Voice AI Employee
Training builds your call flow from the way your team already handles calls. Ema reads your past call transcripts and standard operating procedures, learns the common patterns in them (the topics callers raise, the steps your team follows, the rules they apply), and generates a complete call flow from what it finds: the Call Stages, the Instructions and guardrails for each one, and the right tools wired up. It then places automated test calls against that flow, scores how it did, and fixes what didn't work.
You don't always need to train. Training is most useful for complex use cases: many different call types, multi-step procedures, or a flow that needs several Call Stages. For something straightforward, like answering common questions, taking a message, or routing callers to the right team, filling in Goals and tasks in the setup wizard is enough to start taking calls.
What data to provide
| Data type | What it is | How it is used |
|---|---|---|
| Call & chat transcripts | Recordings transcripts, chat logs, exports of past conversations. | The primary evidence. Autopilot learns the real topics callers raise and how your team handles them. |
| SOP & reference documents | Standard operating procedures, playbooks, policy docs. | The authoritative procedure. An uploaded SOP takes priority over what the transcripts imply. |
Accepted file types include .txt, .md, .csv, .json, .pdf, .docx, .doc, .html, up to 50 MB per file. Scanned image-only PDFs with no text layer, and .zip archives, are not accepted. Training data is optional, but a useful run needs at least some transcripts or an SOP.
How training runs
A training run typically takes a few minutes, and you don't have to wait on it. While a run is going, a banner appears at the top of the AI Employee with a progress bar, so you can see how far along it is and what it's working on. The run moves through these stages:
| Stage you see | What is happening |
|---|---|
| Reading your data | Loading and segmenting your transcripts and documents. |
| Learning the topics | Identifying the scenarios callers raise and the procedure your team follows. |
| Building the call flow | Writing the Call Stages, Instructions, and tool wiring. |
| Testing the call flow | Placing automated simulated calls against the flow and scoring each one. |
| Refining from test results | Fixing what the test calls got wrong, then re-testing. |
| Completed | A new call flow is ready to review. |
When a run finishes, the banner tells you a new call flow is ready. You can place a test call against it to hear how it handles a conversation before you commit to it.
Applying and retraining
Training never changes your live AI Employee on its own. The generated flow is a draft until you choose Apply to AI Employee, which publishes it as a new version. Until you apply, the live AI Employee keeps running exactly as before.
Retraining is manual. Upload new data or make changes, then start a new run. Each run regenerates the whole flow from the current data rather than patching the previous version, so:
- To stop bad data from affecting future runs, remove that file. Deletion takes effect on the next run.
- Manual configuration edits made since the last training run may not be preserved when you retrain and apply. Ema shows a reminder when your data or configuration has drifted since the last run.
Because every published version is retained, you can return to an earlier version at any time. Calls already in progress always finish on the version they started on; only new calls pick up a newly applied version. See Versioning.
Advanced settings
After the wizard, the AI Employee's Advanced settings tab exposes how the AI Employee sounds and what it stores. Changes take effect when you publish a new version; a call already in progress is not affected.
Voice
- Voice: the voice the AI Employee speaks with. Browse the catalog, and preview a voice to hear a sample before you choose.
Conversation behavior
- Barge-in interruption: let callers talk over the AI Employee, which stops speaking to listen. On by default. Turn it off only for lines so noisy that background sound keeps cutting the AI Employee off.
- End call on silence: automatically hang up after a stretch of silence, so a call the caller has walked away from doesn't stay open. Turn it on and set how many seconds of silence to wait before hanging up.
Background sounds
Ambient audio that makes the AI Employee sound like a person at a desk. Both are on by default; turn either off for a silent line.
- Background ambience: play a faint office murmur underneath the call.
- Typing sounds: play keyboard sounds while the AI Employee is looking something up, so silences feel intentional rather than like a dropped call.
Compliance and data retention
- Store audio recordings: keep an audio recording of each call to review in the Audit tab. On by default.
After-call work
Define the information to pull from every call, and where it goes.
- Post-call extraction: declare the fields you want extracted from each conversation. Each field has a name, a type (Text, Selector with a fixed list of options, Boolean, or Number), an optional description, and whether it is required. A plain-language call summary is always produced, even with no fields declared. You can define up to 50 fields.
- After-call workflow: send the extracted data to a companion Dashboard AI Employee that runs your after-call actions, like updating records or triggering follow-ups. Off by default; the companion workflow is created for you when you turn it on.
More controls are on the way. Additional voice tuning, including pronunciation, keypad-input options, and call-length limits, is being finalized and will appear in Advanced settings as it ships.
Call Stages
A Call Stage is one phase of a call. It has its own Instructions (what to do right now), its own set of tools, and its own voice and speech model. While a call is in a stage, the AI Employee is trying to accomplish that stage's job, using that stage's tools, until a condition sends it somewhere else.
Call Stages are what let one Voice AI Employee handle a whole call center's worth of situations without becoming one giant, confused set of Instructions. Each stage stays focused on one job.
The simplest AI Employee is one Call Stage
If your AI Employee does one thing (answer FAQs, take a message), one Call Stage handles the entire call. It greets the caller, answers, and ends. You do not need multiple stages to get value.
How a call moves through stages
- The call starts at the first stage, which plays the welcome message.
- The AI Employee moves to another stage when the conversation calls for it. You give each stage a transition to the stages it can lead to, with a short description of when to take it. The AI Employee reads those descriptions and routes silently, with no menu-reading delay.
- It remembers what the caller has already said. A caller who verified their identity or gave an order number earlier in the call isn't asked again in a later stage.
- Getting back is automatic. Every stage past the first can return to the start ("main menu") or to the previous stage without you wiring those paths by hand.
- The call ends when the AI Employee hangs up (its job is done) or transfers the caller to a person.
Building Call Stages in the builder
You add and connect Call Stages visually in the AI Employee builder. Each Call Stage is a node, and the transitions between stages are the connections you draw between them.
- Add a Call Stage for each phase of the call, and give it a name.
- Select a stage to open its settings and configure it: Instructions, first message, tools, voice, turn-taking, and its transitions.
- Connect stages by drawing a transition from one to another, with a short description of when the AI Employee should take it.
- The starting stage is where every call begins, and it plays the welcome message.
It works like the rest of the AI Employee builder, so laying out stages and wiring the paths between them is familiar.
What you can customize on a Call Stage
| Option | What it controls |
|---|---|
| Instructions | What the AI Employee should do while in this stage, including how to use what the caller has already provided earlier in the call. |
| First message | What the stage says the moment it is entered. Leave it blank for a stage that should simply continue the conversation. |
| Tools | The actions available in this stage: call an integration or MCP tool, search a Knowledge Base, hand work to a connected AI Employee, capture keypad digits, transfer to a human, or end the call. |
| Transitions | The other stages this stage can move to, each with a description of when to go there. |
| Voice and speech model | Each stage can use a speech-to-speech model (lowest latency, best for natural back-and-forth) or a speech-to-text then text-to-speech pipeline (more inspectable). The choice is per stage, and switching stages mid-call does not drop the caller. |
| Turn-taking | How the stage detects when the caller has finished speaking and how readily it can be interrupted. |
Auditing calls
The Audit tab is where you review what happened on real and test calls. Use it to spot where the AI Employee needs better Instructions, an extra transfer destination, or a fix to a tool.
Browsing calls
The Audit tab lists every call the AI Employee handled, both browser test calls and phone calls. Each row shows the call id, call type, start time, duration, whether it was transferred, who ended the call, a short summary, and its status. You can filter by:
- Call type: all calls, production (phone) only, or test only.
- Disconnected by: the caller, the AI Employee, or a system issue.
- Transfer status: transferred to a human, or not.
- Date range, and free-text search across the call id and transcript.
You can export the filtered list, including review verdicts and comments, as CSV.
Inside a single call
Open a call to see two views:
- Summary: the extracted fields, the plain-language call summary, and your review controls.
- Recording and transcript: the full transcript, redacted for privacy, with an audio player that follows along with the recording (when recordings are stored). Markers show where the call was transferred to a human or disconnected.
You can leave a Pass or Fail verdict with notes on the whole call, and add comments on the call or on an individual turn, so a team can work through calls that need attention.
Privacy
Transcripts are stored redacted, caller phone numbers are never stored in the clear, recording playback links are short-lived and every access is logged, and the Audit tab is limited to builder roles. See PII and governance.
Metrics
The Metrics tab is a dashboard of how the AI Employee is performing over a date range you choose. It updates after each call completes (it is not a live in-progress gauge).
| Metric | What it measures |
|---|---|
| Total Calls | Calls handled by this AI Employee in the period. |
| Unique Callers | Distinct phone callers. |
| Average Handle Time | Average call duration, in minutes. |
| Total Call Duration | Cumulative time spent on calls, in minutes. |
| Deflection Rate | The share of completed calls resolved without transferring to a human. |
Trend charts show calls, average handle time, deflection rate, and unique callers per day. A Calls by Disconnection Reason breakdown shows who ended calls (caller, AI Employee, transfer), and Peak Concurrent Calls shows the busiest moments (on a short delay).
Metrics tell you the shape of your call volume and how often calls resolve without a human. To judge quality, use the Pass/Fail reviews in the Audit tab, which capture your team's own judgment of individual calls.
Going live and what's next
Before you publish, place a browser test call to check the voice, greeting, and flow, and add at least one phone number so callers can reach the AI Employee. Every published configuration is versioned, so you can review and roll back to a previous state.
- Build your first workflow: the workflow building blocks that power connected AI Employees and after-call actions.
- Writing effective Instructions: get more out of each Call Stage's Instructions.
- Knowledge Base setup: give your Voice AI Employee documents to answer from.
- Designing human-in-the-loop forms: collect and review input in after-call workflows.