Chat Metrics
Metrics for chat AI Employees transform raw conversation data into actionable insights. They provide real-time monitoring, user behavior analysis, and performance optimization tools to help you continuously improve the effectiveness of your AI Employees.
Benefits
- Real-Time Performance Monitoring -- Measure usage and adoption of end-to-end workflows, including external tool calls.
- Monitor Feedback -- Track positive and negative feedback rates to assess user satisfaction.
- Pattern Discovery -- Identify the most relevant user actions and workflows to guide future integrations.
- Optimization Insights -- Track workflow reliability and success rates. Spot high error or drop-off points to prioritize UX and reliability improvements.
Supported Channels
Chat metrics cover conversations across all chat channels an AI Employee is deployed to:
- Web -- Embedded chatbot and in-app chat.
- Microsoft Teams
- Google Chat
Key Features
The Metrics Dashboard provides two categories of insights:
1. Primary Metrics
| Metric | Description |
|---|---|
| Total Active Users | Users who sent at least one message during the selected time period. |
| Total User Messages | Number of messages sent by users. |
| Total Sessions | Session counts based on platform rules. Microsoft Teams / Google Chat: new session after 24 hours of inactivity. Web: new session after 30 minutes of inactivity. |
| Positive Feedback Rate | Share of interactions that were not rated negative. Computed as 1 − (negative / (positive + negative + unrated)), so both explicit thumbs-up and interactions left unrated count as non-negative. |
| Total Tool Calls | Total number of tool calls made. |
2. Trend Analysis Over Time
- Active Users Over Time -- Track unique user engagement trends.
- User Messages Over Time -- Monitor message volume trends.
- Sessions Over Time -- Understand session creation patterns based on inactivity rules.
- Positive Feedback Rate Over Time -- Measure changes in satisfaction levels.
- Tool Calls by Status Over Time -- Analyze completed, failed, and abandoned tool calls.
- Tool Calls by Tool -- Breakdown of tool usage by integration.
Filtering and Timeframes
All metrics are displayed in Coordinated Universal Time (UTC). The date range picker supports custom start and end dates as well as the following presets:
- Today
- Last 7 full days
- Last week
- Last 14 full days
- Last 30 full days
- Last month
- Last 365 full days
Note: "Last X full days" presets include completed days only and exclude today, so intra-day data does not skew comparisons.
How to Use
- The Metrics tab is automatically enabled for all chat-based AI Employees.
- No additional configuration is required to start tracking metrics.