Testing AI Employees
Testing an AI Employee (AIE) means running its workflow with real or representative inputs and checking what it produced. You do this throughout the build, not just at the end — run a partial workflow to confirm one branch works, run the whole thing to confirm an end-to-end response, and run it in test mode when you don't want to touch live integrations.
An AI Employee is a workflow: a directed graph of typed agent nodes. Running the AIE triggers a workflow run, which executes the nodes in order and records a full trace you can inspect afterward in debug logs. This page covers single, interactive runs.
Ways to run an AI Employee
How you run an AIE depends on its interaction type:
- Chat AI Employees run as a conversation. You open a chat session and send messages; each message triggers a workflow run scoped to that session.
- Non-chat AI Employees (for example document- or form-triggered) run from an input form. You fill in the workflow's input parameters and submit; that triggers a single run.
Both are available from the AI Employee builder — open the AIE, then choose to run it. You stay inside the builder, so you can edit a node and re-run without leaving the page.
Run a chat AI Employee
- Open the AI Employee and start a run. The chat opens directly — the conversation is never gated behind inputs. If the workflow declares input parameters, an inputs control in the chat header lets you provide them; the values are applied when your first message creates the session.
- Type a message and send it. The response streams back in real time.
- Use the session list to switch between conversations, start a new chat, or rename and delete sessions. Each session keeps its own history, so you can test follow-up handling (does the AIE retain context across turns?) and context isolation (does an unrelated follow-up leak the previous topic?).
Run a non-chat AI Employee
- Open the AI Employee and start a run.
- The run form shows the workflow's input parameters, derived from the workflow's input schema. Fill them in.
- Submit to trigger one run. When it completes, the result and full step trace are available.
Test mode (dry run)
Test mode runs the workflow as a dry run: the same graph, agents, and LLM calls execute, but when an agent would invoke an integration Tool (action_*, mcp_*) or a document write (create_document, update_document), the call is short-circuited before it reaches the external system. The agent receives a static acknowledgment echoing the tool name and the arguments it would have been called with. Knowledge-base search is deliberately let through because it is read-only — retrieval behavior stays real in a dry run.
Use test mode to:
- Validate workflow logic, branching, and agent wiring — including that agents pick the right Tools and fill the right arguments — before you want to touch live integrations or spend real API quota.
- Avoid creating real records in downstream systems from agent Tool calls (no tickets or writes).
Two limits to know. The mocked Tool response is an echo of the call's arguments, not realistic output data — a downstream agent that depends on real Tool output sees the acknowledgment instead, so test mode proves wiring and argument-filling, not output handling. And the first-class send-email node is not short-circuited: a dry run that reaches a send-email node sends a real email. Keep test runs away from that node, or point it at a test recipient.
Test mode is an API-level capability today — there is no test-mode toggle in the chat composer. Start a dry run with POST /workflows/{id}/dry-run, or send a chat message with the per-message is_dry_run flag, which you can flip between turns in the same session. In the builder, Autopilot's test-run action starts dry runs the same way. The run is recorded with is_dry_run: true, and its debug logs are as detailed as any other run's.
A note on metrics: dry runs are excluded from the workflow-run product metrics (a dry run emits no run-started event), but chat messages sent in test mode are stored like any other message and are not filtered out of the chat metrics exports.
When mocks aren't enough. A dry run's mocked Tool responses only acknowledge the call — they don't return realistic data. To validate against real systems safely, point your Tools at sandbox or non-production instances of the connected app and run with test mode off. Switch to production instances only at launch — see Launching and monitoring.
Under the hood: runs and the API
Interactive runs go through the workflow service. The same endpoints are available in the API reference if you want to drive runs programmatically.
| Action | Endpoint | Notes |
|---|---|---|
| Start a run | POST /workflows/{id}/run | Body is StartRunRequest (input_params, optional session_id). Returns the run asynchronously. |
| Start a dry run | POST /workflows/{id}/dry-run | Same request shape; the run is marked as a dry run (is_dry_run: true). Unlike /run, it does not link the run to a session_id, even when one is supplied. |
| List runs for a workflow | GET /workflows/{id}/runs | Filter by trigger_type (ui, api, schedule, webhook, feedback, document, app_public, email), status, and since. |
| Get a run | GET /runs/{run_id} | Full detail including token usage and is_dry_run. |
| Get a run's step trace | GET /runs/{run_id}/steps | The ordered list of steps — what debug logs renders. |
| Cancel a run | POST /runs/{id}/cancel | Aborts a pending, running, or paused run and cancels its pending HITL requests. Admin only. Returns 409 if the run is already terminal. |
A run moves through these statuses: pending, running, paused (waiting on a human-in-the-loop request), completed, failed, or cancelled. A seventh status, skipped, marks a scheduled fire the scheduler skipped (recorded with a skip_reason — for example, the previous scheduled run was still active).
POST /workflows/8f3c.../run
{
"input_params": {
"query": "What's our PTO carryover policy in California?"
}
}
Canceling a stuck run
If a run is taking too long or is paused on a human-in-the-loop step you no longer want to answer, an admin can cancel it. Cancellation transitions every pending HITL request on the run to cancelled in one step and marks the run cancelled. Runs that have already finished (completed, failed, cancelled) cannot be canceled and return a conflict.
What to test
A practical checklist before you consider an AIE ready to evaluate or launch:
- Happy path. The most common inputs produce correct, complete responses.
- Branching. Each branch of the workflow is reachable and selects correctly. Use the step trace to confirm which branch ran.
- Follow-ups (chat). The AIE retains relevant context and does not leak unrelated context across turns.
- Out-of-scope inputs. The AIE handles inputs it should decline or redirect, rather than hallucinating an answer.
- Human-in-the-loop. Steps that pause for approval, a form, or a question behave correctly and resume after a response.
- Tools. With test mode off and sandbox integrations connected, Tools call the right systems with the right arguments.
What's next
- Debug logs — inspect exactly what a run did, step by step.
- Launching and monitoring — publish and roll out once testing is done.