1. Ema v2 Docs
  2. Testing Operations
  3. Launching And Monitoring

Launching and Monitoring

Launching an AI Employee means publishing a workflow version so it goes live for your Workspace, then watching how it performs with real users. Ema is a managed cloud platform. There is no deploy step and no infrastructure to provision. You publish, the new version becomes the one users hit, and the built-in metrics start tracking it.

This page covers publishing a version, rolling out responsibly, and the monitoring surfaces available afterward.

Publishing a version

A workflow has a working draft and a series of published versions. Publishing promotes the current state to a new live version with POST /workflows/{id}/publish.

Publishing runs validation first. If the workflow can't go live, publish fails with a validation error (422) instead of shipping something broken. Validation covers, among others:

  • Structural DAG errors (disconnected or malformed agents).
  • LLM-provider and model checks.
  • Human-in-the-loop assignee checks.
  • MCP server and per-tool configuration errors.
  • Agent-type version pins that are deprecated, deleted, broken, unknown, or pre-release.

Fix the reported issues and publish again. You can list past versions with GET /workflows/{id}/versions. To roll back a launch, promote a previously published version: promotion re-activates it immediately. (Revert is different: it stages an older version's state as a new draft without changing what is live, so it only takes effect once you publish that draft.)

Before you launch

Treat launch as the end of the testing path, not the start. Confirm:

  • Quality is measured, not assumed. Run an evaluation over a representative dataset and confirm the scores meet your bar, especially the low-percentile tail, which surfaces the worst-case responses.
  • Integrations point at production. If you tested against sandbox instances or with test mode, switch your Tools to the real production systems before users rely on them.
  • No test artifacts remain. Remove any debugging agents or temporary outputs you added while building.
  • Name and description are user-ready. They appear to the people who use the AI Employee.
  • Access is set up. Confirm the right users and admins have access. See Administration for roles and permissions.

Roll out in phases

A controlled rollout limits the blast radius of anything you missed in testing and makes feedback manageable.

  1. Start small. Launch to a limited group first: enough users to surface real-world patterns, few enough that you can respond to each piece of feedback.
  2. Focus on high-impact cases. Prioritize the use cases that deliver the most value so early users see the benefit quickly.
  3. Define phases. For each phase, set the audience, timeline, and exit criteria (what has to be true before you widen access).
  4. Collect and triage feedback. Give users a clear channel to report problems, distinguish bugs from feature requests, and review submissions on a regular cadence. In-product thumbs-up/thumbs-down feedback also feeds the metrics below.

For chat AI Employees that you embed elsewhere (a website, a Microsoft Teams or Google Chat workspace), the rollout is the embed itself. See Integrations & data for channel setup.

Monitoring after launch

AI Employees get a Metrics tab in the builder, with no extra configuration (the one exception is orchestrator AI Employees, whose fixed tab set has no metrics). What it shows depends on the AI Employee's interaction type:

  • Chat AI Employees get a conversational analytics dashboard: usage, sessions, messages, active users, feedback trends, and tool-call activity, rendered from reusable dashboard templates you can instantiate, customize, and save.
  • Voice AI Employees get a dedicated voice analytics dashboard: call volume, deflection, disconnection reasons, concurrency, handle time, and unique callers.
  • All other AI Employees (form-, document-, or schedule-triggered, recruiter, and other non-chat types) share a generic execution dashboard: rows processed, succeeded, and failed, run latency, run cost, and agent and Tool activity.

The dashboard is built on Ema's BI analytics layer: you pick a template, instantiate it for the AI Employee, and can drill down between linked dashboards. What you can edit depends on your role. You can also change what the dashboard shows: add, edit, move, and remove charts, then publish the result. See Building a custom metrics view.

Schedules tab

The Schedules tab on the Metrics page lists all scheduled workflow runs across the entire Workspace, alongside higher-level notes on the overall success rate of schedule triggers so you can monitor their health at a glance. You can filter by status, including Deleted, to find schedules that have been removed and audit when they last ran. Available status filters: Active, Paused, Completed, Failed, and Deleted.

Queued runs

When the system is under load, workflow runs may be held in a queue before they start. A queued row on the AI Employee dashboard now shows its position in the queue and the reason it is waiting. You can cancel a queued run directly from the dashboard row without waiting for it to start.

Per-AI-Employee run limits

Ema can also cap how many runs of a single AI Employee execute at the same time. This is a platform setting that Ema configures for your workspace on request; it is not available in Workspace Management. By default there is no per-AI-Employee limit, so only the workspace-wide limit applies. When a per-AI-Employee limit is in place:

  • Runs that exceed the limit are held in the queue and start as earlier runs finish. They are not rejected.
  • The queue position shown for a waiting run does not account for the per-AI-Employee limit. The position reflects the run's place in the workspace queue, so the actual wait may be longer if the AI Employee itself is at its limit.
  • Chat runs for the same AI Employee are not affected by this limit.

If runs for one AI Employee queue longer than expected while others start promptly, contact Ema support to check whether a per-AI-Employee limit is set.

Paused AI Employees and the dashboard tab

The header toggle in the AI Employee builder pauses the AI Employee independently of its publish status. When an AI Employee is paused, no new run can start through any entry point. In the dashboard tab this means:

  • The Add button is disabled.
  • The per-row Play (rerun) button is disabled.
  • The Run Selected bulk-rerun button is disabled.

Each disabled control shows a tooltip: "This AI Employee is disabled. Enable it from the header toggle to run rows." Editing a row's text inputs and viewing Show Work or human-in-the-loop details are unaffected by the pause. If a row edit carries a file attachment, that edit is also blocked, because attaching a file starts a run.

To start runs again, re-enable the AI Employee using the header toggle.

Reviewing real conversations

Beyond aggregate charts, you can review individual conversations a chat AI Employee has had: read the full message history with the feedback users left on each answer, open the execution trace behind an answer by clicking the Show work button on the row, then mark a conversation's resolution status and leave reviewer notes. This conversation-review surface is admin-gated and is the qualitative companion to the quantitative dashboard. The dashboard tells you what changed, the conversations tell you why.

Exporting raw data

When you need the underlying rows in your own warehouse or BI tool, not just the in-app charts, export them. The dashboard JSONL export is delivered as a zip file. Chat AI Employees also support row-level exports: messages as CSV, NDJSON, or JSON, and the legacy conversations contract as CSV or JSON. See Chat metrics for the full export contract, columns, and filters.

Closing the loop

Monitoring isn't passive. The intended cycle:

  1. Watch the dashboard for drops in feedback rate, spikes in errors, or unexpected tool-call patterns.
  2. Open the affected conversations or debug logs to see what went wrong.
  3. Reproduce and fix in the draft, test the fix, and evaluate it against your dataset.
  4. Publish the new version and keep watching.

What's next

Last updated: Oct 12, 2026