Building a Voice AI Employee

A Voice AI Employee answers phone calls and talks with callers in real time. It listens, speaks, looks things up, takes actions in your systems, and hands off to a person when a call needs one. A caller reaches it by dialing a phone number, and you can also place a test call from your browser while you build.

This guide is the complete builder walkthrough: why Ema Voice is built the way it is, how a call is structured, the setup wizard, how training builds your call flow from real calls, the advanced settings, Call Stages (the building block for complex flows), and the audit and metrics you use to watch it in production.

Why Ema Voice

Ema Voice is built to feel like a person, act on your real systems, and stay accountable for every call.

• Realtime and cascaded speech pipelines. Each Call Stage can run a single realtime speech-to-speech model, a realtime model paired with a dedicated voice (the default for new AI Employees), or a full speech-to-text, language model, and text-to-speech pipeline when you want control over each step. You choose per stage, in the same call.
• Runs complex work while it talks. During a call it can trigger other AI Employees in the background to run multi-step workflows (look up an account, file a claim, update a record) and bring the result back into the conversation.
• Built from your past calls, and keeps improving. An automated training pipeline reads your real call transcripts and standard operating procedures, builds the call flow, then runs automated test calls to score and refine it. After launch, the calls you flag in the Audit tab feed targeted improvements.
• Talks at a human pace, and callers can always interrupt. Callers can talk over the AI Employee and it stops to listen. If a caller changes their mind mid-sentence ("actually, ticket 4471, not 4417"), the correction takes, and it doesn't read back the answer the caller already moved past.
• Every call is auditable. For each call you can review the transcript synced to the recording, every tool the AI Employee called with its inputs and results, the fields that were extracted, when it transferred or ended the call, and your team's reviews.

How a Voice AI Employee works

A Voice AI Employee is a normal AI Employee with its interaction type set to Voice. It runs on a different framework from the workflows behind chat and dashboard AI Employees, and that difference is worth understanding before you build.

A call is a flow of Call Stages

Each Call Stage is one phase of the conversation, with its own Instructions, its own tools, and its own voice and speech model. While a call is in a stage, the AI Employee works on that stage's job until the conversation moves on. The simplest Voice AI Employee is a single Call Stage that handles the whole call; a complex one routes between many.

The AI Employee's Goals and tasks and its Persona and guardrails apply to every stage. Each stage's own Instructions describe only that step of the call, so you set the rules once and they hold no matter where the conversation goes.

Your workflows run as tools, mid-conversation

Call Stages don't replace your workflows. At any point in a call, the AI Employee can trigger a workflow (by handing work to another AI Employee) as a tool call. It runs in the background while the AI Employee keeps talking to the caller, and the result comes back into the conversation. This is how a call does real work, like looking up an account, filing a claim, or updating a record, without leaving the caller in silence.

Two limits to know: a workflow that pauses for human input can't complete mid-call (the AI Employee is told it couldn't finish and carries on with the caller), and a tool call has about a minute to return before the AI Employee moves on.

Why voice is built this way

Latency is the constraint that matters most on a phone call. Callers notice a pause of even half a second, so every conversational turn has a very small budget. A workflow runs a fixed graph of Agents from start to finish, which is the right model for chat and batch work, but on a call it adds delay to every turn and forces the conversation down a predetermined path. Call Stages let the AI Employee answer immediately and reach for a full workflow only at the moments that actually need one.

Setting up your Voice AI Employee

In the Create AI Employee dialog, give the AI Employee a Name and a Description (both required), set Interaction Type to Voice, and optionally pick a Category. Choose Create.

Ema provisions the voice configuration automatically and opens a setup wizard that walks you through four steps: Basic settings, Connections, Telephony settings, and Training data. You can move back to an earlier step at any time; only the first step has a required field, and your progress is saved in the browser as you go.

Step 1: Basic settings

This step defines who the AI Employee is and how it speaks. Everything here applies to the whole call, across every Call Stage.

Step 2: Connections

Connect what the AI Employee can draw on during a call. All three are optional and can be added later.

• Knowledge Bases: the documents the AI Employee can search to answer caller questions, like policies, product details, or an FAQ. Connected Knowledge Bases also inform training, which decides where in the call flow a knowledge search belongs. See Setting up a Knowledge Base.
• Connected AI Employees: other AI Employees this one can hand work to. They run their specialized workflows behind the scenes and return a result to the call.
• Tools: actions the AI Employee can take on the caller's behalf, drawn from your Workspace's connected integrations and MCP servers. See the Integrations Hub.

Step 3: Telephony settings

This step connects real callers to the AI Employee and defines where it can transfer them.

Inbound number

The Telephony integrations card lists the numbers that route inbound calls to this AI Employee. Choose Add number to open the Add a phone line dialog, which offers three paths:

Everything behind the number (the SIP trunk and call routing) is provisioned for you. A number shows as Provisioning while it is being set up and Active once it can take calls.

Agent handoff

The people and phone numbers the AI Employee can transfer a live call to. Choose Add handoff and fill in:

• Label: a name for the destination, for example "Billing team".
• Destination: where to send the call, as a tel: phone number or a sip: address (for example tel:+14155551212 or sip:[email protected]). A bare number isn't accepted.
• When to hand off: a short description the AI Employee reads to decide when this destination applies.
• Set availability hours (optional): a timezone, the days, and opening and closing times when this destination is staffed, with more than one window if you need a split shift. Outside those hours the AI Employee doesn't transfer; it tells the caller when to call back. Destinations with hours show an Hours badge on the card.

A transfer is warm. The AI Employee tells the caller it's connecting them, dials the person into the same call, and if nobody answers within about 45 seconds it picks the conversation back up and offers to take a message instead of dropping the caller. This works on phone calls and on browser test calls.

Adding a destination doesn't by itself make every stage able to transfer. You switch Transfer to human on for the specific Call Stages where a handoff makes sense, and optionally pin a stage to one destination. See Call Stages.

Step 4: Training data

Add past call transcripts and reference documents, and Ema builds the call flow from them. Two upload areas are offered: Past call & chat transcripts and SOP & reference documents. See Training your Voice AI Employee for what to upload and how it's used.

Finish the wizard with Train AI Employee (available once you've added at least one file) or Skip for now. Either way, the AI Employee is created and you land on its Configuration tab. You can add training data later from Configuration, under Past call & chat transcripts and Reference documents.

Training your Voice AI Employee

Training builds your call flow from the way your team already handles calls. Ema reads your past call transcripts and standard operating procedures, learns the common patterns in them (the topics callers raise, the steps your team follows, the rules they apply), and generates a complete call flow from what it finds: the Call Stages, the Instructions and guardrails for each one, and the right tools wired up. It then places automated test calls against that flow, scores how it did, and fixes what didn't work.

What data to provide

Accepted file types are .txt, .md, .csv, .json, .pdf, .docx, .pptx, .xlsx, .html, and .htm, up to 50 MB per file. Legacy .doc files, scanned image-only PDFs with no text layer, and .zip archives are not accepted. Either kind of data is enough on its own: an SOP with no transcripts still produces a working flow. When you have many transcripts, the newest 50 files are used for each run.

Knowledge Bases connected on the Connections step are also read during training, so the call flow knows which stages should search your documents.

How training runs

A quick improvement or voice check takes a few minutes; a full training run can take tens of minutes. You don't have to wait on it. While a run is going, a status card appears above the tabs on the Configuration and Audit views with a progress bar and the current stage. The run moves through these stages:

While a run is in progress, configuration is locked: you can't save edits until it finishes or you choose Cancel training. As soon as the draft flow exists (after "Saving the draft"), a Test AI Employee button appears so you can place a browser call against the new flow while the automated test calls are still scoring it.

When a run finishes, the card tells you a new call flow is ready and offers Test AI Employee and Apply to AI Employee. If the automated evaluation didn't finish but the flow was built, the card says so and still lets you test and apply the draft. If training fails outright, nothing about the live AI Employee changes.

Applying and retraining

Training never changes your live AI Employee on its own. The generated flow is a draft until you choose Apply to AI Employee, which publishes it as a new version. Until you apply, the live AI Employee keeps running exactly as before.

Retraining is always your decision, but Ema watches for the reasons you'd want to and offers the right-sized run in a banner at the top of the AI Employee:

Cosmetic edits (the welcome message, persona wording, voice choice, language, recording and silence settings) don't trigger a banner. Each banner asks you to confirm before it starts, and where a scoped run is offered you can choose Rebuild the whole flow instead. You can't start a run while you have unsaved edits.

A full retrain uses your currently published flow as its starting point, so stages whose job hasn't changed are updated rather than replaced. Rebuilding the whole flow rewrites the entry stage, which means your welcome message may be regenerated; check it after applying.

Every published version is kept. Open Version history in the builder and choose Make active on an earlier version to roll back. Calls already in progress always finish on the version they started on; only new calls pick up the newly active version. To stop a bad file from affecting future runs, remove it from Configuration; the removal takes effect on the next run. See Versioning.

Advanced settings

After the wizard, the AI Employee's Configuration tab has three sections: Basic settings, Telephony settings, and Advanced settings. Choose Edit in the header to make changes and Save to apply them. A call already in progress is not affected; new calls pick up the saved settings.

Voice model

Choose how the voice model is picked, and the voice your AI Employee speaks with.

• EmaFusion™ (default, recommended): Ema picks the best voice model for every call. You choose only the Voice, from a dropdown you can search by name, gender, or tone, with a preview of each voice.
• Limit to a specific model: pin the pipeline and model yourself. Pick a Model pipeline (Realtime, where one model handles speech in and out for the lowest latency, or Cascaded, with separate speech-to-text, language, and text-to-speech models for more control over each step), then the model and the voice.
• Custom model: coming soon.

The voice you choose here applies to every Call Stage unless a stage overrides it in the builder.

Voice activity detection

Tune how the AI Employee listens, responds, and ends calls. Callers can always talk over the AI Employee and it stops to listen; there is no setting to turn that off.

• Allow keypad input: let callers enter numbers on their phone keypad instead of saying them, for things like account numbers or PINs. Off by default. When on, choose the Default submit key (# or *) callers press to finish an entry.
• End call on silence: hang up automatically when a caller has gone quiet, so an abandoned call doesn't stay open. Off by default. When on, set Seconds of silence before hang-up (5 to 300, default 10) and Check-ins before hang-up (0 to 5, default 2): the AI Employee checks in that many times first, the last time warning that the call is about to end, and any reply from the caller resets the count. Set check-ins to 0 to hang up at the first silence.

Background sounds

Ambient audio that makes the AI Employee sound like a person at a desk. Both are on by default; turn either off for a silent line.

• Background ambience: play a faint office murmur underneath the call.
• Typing sounds: play keyboard sounds while the AI Employee is looking something up, so silences feel intentional rather than like a dropped call.

Compliance and data retention

• Store audio recordings: keep an audio recording of each call to review in the Audit tab. On by default.

After call work

Define the information to pull from every call, and where it goes.

• Post call extraction: declare the fields you want extracted from each conversation. Each field has a name, a type (Text, Selector with a fixed list of options, Boolean, or Number), an optional description, and an Optional field switch (turn it off to require the field on every call). A plain-language call summary is always produced, even with no fields declared. You can define up to 50 fields.
• After call workflow: send the extracted data to a companion Dashboard AI Employee that runs your after-call actions, like updating records or triggering follow-ups. Off by default. When you turn it on and save, Ema creates the companion AI Employee for you and reminds you to build its workflow; nothing runs until you do. Use Open AI Employee to go straight to it.

Call Stages

A Call Stage is one phase of a call. It has its own Instructions (what to do right now), its own set of tools, and its own voice and speech model. While a call is in a stage, the AI Employee is trying to accomplish that stage's job, using that stage's tools, until a condition sends it somewhere else.

Call Stages are what let one Voice AI Employee handle a whole call center's worth of situations without becoming one giant, confused set of Instructions. Each stage stays focused on one job, while your AI Employee's Goals and Persona and guardrails apply throughout.

The simplest AI Employee is one Call Stage

If your AI Employee does one thing (answer FAQs, take a message), one Call Stage handles the entire call. It greets the caller, answers, and ends. You do not need multiple stages to get value.

How a call moves through stages

• The call starts at the entry point, which plays the welcome message.
• The AI Employee moves to another stage when the conversation calls for it. You give each stage transitions to the stages it can lead to, each with a short description of when to take it. The AI Employee reads those descriptions and routes silently, with no menu-reading delay. A transition can also be given a keypad key, so pressing that key on the phone takes it immediately.
• It remembers what the caller has already said. A caller who verified their identity or gave an order number earlier in the call isn't asked again in a later stage.
• Every stage speaks when the call enters it. It says the stage's first message if you set one, and otherwise opens in its own words, so a transition never sounds like a dropped call.
• Getting back is automatic. Every stage past the entry point can return to the start or to the previous stage without you wiring those paths by hand.
• The call ends when the AI Employee hangs up (its job is done) or transfers the caller to a person.

Building Call Stages in the builder

Choose Open Builder in the AI Employee's header to open the visual builder. Each Call Stage is a node, and each transition is a connection drawn from one stage to another.

• Add conversation stage adds a new stage. It gets a placeholder name; select it and use the pencil in the panel header to rename it.
• Select a stage to open its settings panel next to the stages (see the next section).
• Connect stages by dragging from a stage's right-hand handle onto another stage, or by adding a transition in the panel and picking a Target stage (you can create the target stage right there).
• The entry point is where every call begins. It carries an Entry point badge, you can move it to another stage with Make entry point from the stage menu, and it can't be deleted. Its first message is your welcome message.
• The stage menu also offers Duplicate and Delete stage. Deleting a stage removes the transitions that led to it; the builder tells you how many before you confirm.
• Clean up layout auto-arranges the stages left to right. Your layout is kept when you save.
• A Validation issues pill appears when something needs fixing, for example a transition that points at a stage that no longer exists.

Save stores your changes as a new unpublished version, and Publish makes the last saved version live for calls. The header shows Unsaved changes while you're editing, and Test becomes available once your changes are published. Version history lists every version with Live and Draft badges and lets you make an earlier one active. While a training run is in progress you can't save; wait for it to finish or cancel it. Back on the AI Employee's page, a banner reminds you when you have unpublished changes in the builder.

What you can configure on a Call Stage

The settings panel for a selected stage is organized into these sections.

Auditing calls

The Audit tab is where you review what happened on real and test calls. Use it to spot where the AI Employee needs better Instructions, an extra transfer destination, or a fix to a tool, and to flag calls that should improve the call flow.

Browsing calls

The Audit tab lists every call the AI Employee handled, both browser test calls and phone calls, 20 to a page. Each row shows the call ID (shortened), the call type (Test call or Phone call), start time, duration, whether it was transferred, who disconnected (End user, Ema, or System error), a short summary, an outcome, and the call's status (Completed, In progress, Failed, or Abandoned). You can filter by:

• Call type: all calls, production only, or test only.
• Disconnected by: the end user, Ema, or a system error.
• Transfer status: transferred to a human, or not.
• Outcome: Resolved, Partially resolved, Unresolved, or No intent. The outcome is inferred from how the call ended, and this filter applies to the page you're viewing.
• Date range.

The search box, Search call ID or transcript, finds a call by its full ID or by words spoken in the transcript. Export downloads the current filtered list as CSV (up to the newest 50 matching calls) with the call details, summary, and your team's review verdict and notes. Opening a call adds it to the page URL, so you can share a link to a specific call, and you can step to the previous or next call from inside it.

Inside a single call

Open a call to see two views:

• Summary: the call's date, duration, and who ended it, the plain-language call summary, the extracted fields, and the review controls. A Live indicator shows while a call is still in progress.
• Recording and transcript: the full transcript, updated every few seconds while the call is live. When a recording is stored, an audio player follows along: play and pause, skip forward and back, change playback speed, scrub the waveform, or click any line to jump to it. Every tool the AI Employee called appears inline where it happened; switch from Narrative view to Developer view to see each tool's inputs and results and whether it succeeded. A marker at the end shows whether the call was transferred to a human or disconnected.

Reviewing a call

Under Rate this session, mark a completed call Well handled or Could be better and add Notes on what the AI Employee should keep doing or do differently. You can also add a comment on any of the AI Employee's turns in the transcript to say what it should have done at that moment.

These reviews do more than keep a record. Calls marked Could be better, with your notes and comments, become the input to Improve from feedback (see Applying and retraining): Ema patches the parts of the call flow your feedback points at and turns each flagged call into a test it keeps passing.

Privacy

Where PII masking is enabled for your deployment, transcripts are stored with personal information masked. The caller's phone number is never shown in the Audit tab, and only a non-reversible fingerprint of it is used for metrics. Recording playback links expire after 15 minutes and every access is logged. The Audit tab is available to the builder, builder admin, and system admin roles. See PII and governance.

Metrics

The Metrics tab is a dashboard of how the AI Employee is performing, showing the last 30 days by default with a date range you can change. Each call appears within moments of ending, and the headline tiles compare the current period with the one before it.

Trend charts show Calls Over Time, Average Handle Time Over Time, Deflection Rate Over Time, and Unique Callers Over Time by day. Calls by Disconnection Reason breaks calls down by how they ended: hung up by the caller, ended by the AI Employee, transferred, ended after silence, or cut off at the call-length limit. Peak Concurrent Calls shows your busiest moments, about an hour behind real time. You can export the dashboard to a spreadsheet or reset it to the default layout.

Going live and what's next

Before you publish, place a browser test call to check the voice, greeting, and flow, and add at least one phone number so callers can reach the AI Employee. Every published configuration is versioned, so you can review earlier versions and make one active again at any time.

• Build your first workflow: the workflow building blocks that power connected AI Employees and after-call actions.
• Writing effective Instructions: get more out of each Call Stage's Instructions.
• Knowledge Base setup: give your Voice AI Employee documents to answer from.
• Designing human-in-the-loop forms: collect and review input in after-call workflows.