1. Ema v2 Docs
  2. Testing Operations
  3. Launching And Monitoring

Launching and Monitoring

Launching an AI Employee (AIE) means publishing a workflow version so it goes live for your tenant, then watching how it performs with real users. Ema is a managed cloud platform — there is no deploy step and no infrastructure to provision. You publish, the new version becomes the one users hit, and the built-in metrics start tracking it.

This page covers publishing a version, rolling out responsibly, and the monitoring surfaces available afterward.

Publishing a version

A workflow has a working draft and a series of published versions. Publishing promotes the current state to a new live version with POST /workflows/{id}/publish.

Publishing runs validation first. If the workflow can't go live, publish fails with a validation error (422) instead of shipping something broken. Validation covers, among others:

  • Structural DAG errors (disconnected or malformed nodes).
  • LLM-provider and model checks.
  • Human-in-the-loop assignee checks.
  • MCP server and per-tool configuration errors.
  • Agent-type version pins that are deprecated, deleted, broken, unknown, or pre-release.

Fix the reported issues and publish again. You can list past versions with GET /workflows/{id}/versions, and revert to a previously published version if a launch needs to be rolled back.

Pin evaluations to a version. Because each publish creates a numbered version, you can pin an eval config to a specific workflow_version_id and re-run the same dataset against it. That gives you a reproducible quality score for the exact version you're about to launch.

Before you launch

Treat launch as the end of the testing path, not the start. Confirm:

  • Quality is measured, not assumed. Run an evaluation over a representative dataset and confirm the scores meet your bar — especially the low-percentile tail, which surfaces the worst-case responses.
  • Integrations point at production. If you tested against sandbox instances or with test mode, switch your Tools to the real production systems before users rely on them.
  • No test artifacts remain. Remove any debugging nodes or temporary outputs you added while building.
  • Name and description are user-ready. They appear to the people who use the AIE.
  • Access is set up. Confirm the right users and admins have access. See Administration for roles and permissions.

Roll out in phases

A controlled rollout limits the blast radius of anything you missed in testing and makes feedback manageable.

  1. Start small. Launch to a limited group first — enough users to surface real-world patterns, few enough that you can respond to each piece of feedback.
  2. Focus on high-impact cases. Prioritize the use cases that deliver the most value so early users see the benefit quickly.
  3. Define phases. For each phase, set the audience, timeline, and exit criteria (what has to be true before you widen access).
  4. Collect and triage feedback. Give users a clear channel to report problems, distinguish bugs from feature requests, and review submissions on a regular cadence. In-product thumbs-up/thumbs-down feedback also feeds the metrics below.

For chat AI Employees that you embed elsewhere (a website, a Microsoft Teams or Google Chat workspace), the rollout is the embed itself — see Integrations & data for channel setup.

Monitoring after launch

Every AI Employee gets a Metrics tab in the builder, with no extra configuration. What it shows depends on the AIE's trigger type:

  • Chat AI Employees get a conversational analytics dashboard — usage, sessions, messages, active users, feedback trends, and tool-call activity, rendered from reusable dashboard templates you can instantiate, customize, and save.
  • Ticketing AI Employees get a ticketing-oriented metrics view focused on processing volume and resolution.
  • Other trigger types get the metrics appropriate to how they run.

The dashboard is built on Ema's BI analytics layer: you pick a template, instantiate it for the AIE, and can drill down between linked dashboards. What you can edit depends on your role.

Reviewing real conversations

Beyond aggregate charts, you can review individual conversations a chat AI Employee has had — read the full message history, the user context, and the sources it retrieved, then mark a conversation's resolution status and leave reviewer notes. This conversation-review surface is admin-gated and is the qualitative companion to the quantitative dashboard: the dashboard tells you what changed, the conversations tell you why.

Exporting raw data

When you need the underlying rows in your own warehouse or BI tool — not just the in-app charts — export them. Chat AI Employees support row-level CSV, NDJSON, and JSON exports of messages and conversations. See Chat metrics for the full export contract, columns, and filters.

Closing the loop

Monitoring isn't passive. The intended cycle:

  1. Watch the dashboard for drops in feedback rate, spikes in errors, or unexpected tool-call patterns.
  2. Open the affected conversations or debug logs to see what went wrong.
  3. Reproduce and fix in the draft, test the fix, and evaluate it against your dataset.
  4. Publish the new version and keep watching.

What's next

Last updated: Jul 3, 2026