Launching and Monitoring

Launching an AI Employee means publishing a workflow version so it goes live for your Workspace, then watching how it performs with real users. Ema is a managed cloud platform. There is no deploy step and no infrastructure to provision. You publish, the new version becomes the one users hit, and the built-in metrics start tracking it.

This page covers publishing a version, rolling out responsibly, and the monitoring surfaces available afterward.

Publishing a version

A workflow has a working draft and a series of published versions. Publishing promotes the current state to a new live version with POST /workflows/{id}/publish.

Publishing runs validation first. If the workflow can't go live, publish fails with a validation error (422) instead of shipping something broken. Validation covers, among others:

• Structural DAG errors (disconnected or malformed agents).
• LLM-provider and model checks.
• Human-in-the-loop assignee checks.
• MCP server and per-tool configuration errors.
• Agent-type version pins that are deprecated, deleted, broken, unknown, or pre-release.

Fix the reported issues and publish again. You can list past versions with GET /workflows/{id}/versions. To roll back a launch, promote a previously published version: promotion re-activates it immediately. (Revert is different: it stages an older version's state as a new draft without changing what is live, so it only takes effect once you publish that draft.)

Before you launch

Treat launch as the end of the testing path, not the start. Confirm:

• Quality is measured, not assumed. Run an evaluation over a representative dataset and confirm the scores meet your bar — especially the low-percentile tail, which surfaces the worst-case responses.
• Integrations point at production. If you tested against sandbox instances or with test mode, switch your Tools to the real production systems before users rely on them.
• No test artifacts remain. Remove any debugging agents or temporary outputs you added while building.
• Name and description are user-ready. They appear to the people who use the AI Employee.
• Access is set up. Confirm the right users and admins have access. See Administration for roles and permissions.

Roll out in phases

A controlled rollout limits the blast radius of anything you missed in testing and makes feedback manageable.

1. Start small. Launch to a limited group first: enough users to surface real-world patterns, few enough that you can respond to each piece of feedback.
2. Focus on high-impact cases. Prioritize the use cases that deliver the most value so early users see the benefit quickly.
3. Define phases. For each phase, set the audience, timeline, and exit criteria (what has to be true before you widen access).
4. Collect and triage feedback. Give users a clear channel to report problems, distinguish bugs from feature requests, and review submissions on a regular cadence. In-product thumbs-up/thumbs-down feedback also feeds the metrics below.

For chat AI Employees that you embed elsewhere (a website, a Microsoft Teams or Google Chat workspace), the rollout is the embed itself. See Integrations & data for channel setup.

Monitoring after launch

AI Employees get a Metrics tab in the builder, with no extra configuration (the one exception is orchestrator AI Employees, whose fixed tab set has no metrics). What it shows depends on the AI Employee's interaction type:

• Chat AI Employees get a conversational analytics dashboard: usage, sessions, messages, active users, feedback trends, and tool-call activity, rendered from reusable dashboard templates you can instantiate, customize, and save.
• Voice AI Employees get a dedicated voice analytics dashboard: call volume, deflection, disconnection reasons, concurrency, handle time, and unique callers.
• All other AI Employees (form-, document-, or schedule-triggered, recruiter, and other non-chat types) share a generic execution dashboard: rows processed, succeeded, and failed, run latency, run cost, and agent and Tool activity.

The dashboard is built on Ema's BI analytics layer: you pick a template, instantiate it for the AI Employee, and can drill down between linked dashboards. What you can edit depends on your role.

Schedules tab

The Schedules tab on the Metrics page lists all scheduled workflow runs across the entire tenant, alongside higher-level notes on the overall success rate of schedule triggers so you can monitor their health at a glance. You can filter by status, including Deleted, to find schedules that have been removed and audit when they last ran. Available status filters: Active, Paused, Completed, Failed, and Deleted.

Reviewing real conversations

Beyond aggregate charts, you can review individual conversations a chat AI Employee has had: read the full message history with the feedback users left on each answer, open the execution trace behind an answer (Show work), then mark a conversation's resolution status and leave reviewer notes. This conversation-review surface is admin-gated and is the qualitative companion to the quantitative dashboard. The dashboard tells you what changed, the conversations tell you why.

Exporting raw data

When you need the underlying rows in your own warehouse or BI tool, not just the in-app charts, export them. Chat AI Employees support row-level exports: messages as CSV, NDJSON, or JSON, and the legacy conversations contract as CSV or JSON. See Chat metrics for the full export contract, columns, and filters.

Closing the loop

Monitoring isn't passive. The intended cycle:

1. Watch the dashboard for drops in feedback rate, spikes in errors, or unexpected tool-call patterns.
2. Open the affected conversations or debug logs to see what went wrong.
3. Reproduce and fix in the draft, test the fix, and evaluate it against your dataset.
4. Publish the new version and keep watching.

What's next

• Chat metrics — export conversational data for your own analysis.
• Debug logs — investigate a specific run when a metric moves.