Feedback and Continuous Learning
Continuous Learning improves evaluation accuracy over time through reviewer feedback. Unlike Auto-Learning (which uses a batch of golden data during initial setup), Continuous Learning incorporates corrections incrementally as your team reviews evaluations.
7.1 Submitting Feedback
Permission required: CAN_SUBMIT_FEEDBACK (Metrics Viewer, SelfMetrics Viewer for their own interactions, Manager, or Admin)
- In the Audit tab, click View Details on an interaction
- In the Scorecard, find the parameter you want to comment on
- Click the thumbs down icon (disagree with evaluation) or thumbs up icon (agree)
- Enter a feedback comment explaining your assessment
- Click Submit
What Makes Feedback Effective
Good example: "The agent did say 'Team Portugal' at 0:15, which should count as company identification. Ensure that any variation of the company name in the opening counts toward this rule."
Tips: Be specific about what happened in the call (reference timestamps if possible). Explain what the correct evaluation should be and why. Describe the general principle, not just the one-off correction. Avoid vague comments like "This is wrong."
Rules for feedback management:
- Only the feedback creator can update or delete their own feedback
- Updating feedback resets its status to unprocessed, requiring re-moderation
7.2 Moderating Feedback
Permission required: CAN_APPROVE_FEEDBACK (Manager or Admin)
- Navigate to the Audit tab and open the Accepted Feedback view
- Review pending (unprocessed) feedback items
- For each item: read the original evaluation result and rationale, read the feedback comment, check the transcript for context
- Click Accept to approve, or Reject to ignore
Feedback Lifecycle
| Status | Description |
|---|---|
| Unprocessed | Submitted, awaiting moderation |
| Approved | Accepted by a moderator. Eligible for the Continuous Learning pipeline. |
| Processed | Already consumed by the Continuous Learning pipeline. Will not be reused. |
| Ignored | Rejected during moderation. Visible but not used for learning. |
7.3 Generating Improvement Suggestions
Permission required: Manager or Admin.
Once feedback has been approved for a rule, you can ask Ema to suggest an updated rule instruction directly from the Accepted Feedback view:
- Open the Accepted Feedback view. Rules with approved feedback are listed as cards, each showing the rule name, category, description, and a feedback-count badge.
- Click a rule's card to open the Feedback Selection Drawer, listing every approved feedback item for that rule as a checkbox (all pre-selected by default — deselect any you don't want to include).
- Click Suggest improvements.
Ema analyzes the selected feedback and generates an updated rule instruction. A Suggested Improvements modal opens immediately with a loading state ("Generating suggestions... Ema is refining your QA instructions based on the latest feedback") while the pipeline runs.
The pipeline runs through Queued/Running → Completed, Failed, or Timed Out. The modal polls automatically — you don't need to refresh or re-open it. A timed-out run can be re-submitted.
One Improvement Run at a Time, Per Rule
Only one improvement pipeline can run per rule at a time. If another Manager or Admin has already triggered one for the same rule, the drawer shows a banner instead of the checkboxes:
"A pipeline has been started by [name] — please wait for it to complete."
The Suggest improvements button is hidden until that run reaches a terminal state (Completed, Failed, or Timed Out).
7.4 Applying Improvements
When the pipeline completes, the modal shows the original instruction and Ema's suggested instruction side by side — only the pass/fail instruction and the not-applicable (alternate-rule) instruction fields are shown; name, category, and weight are unchanged and not part of the suggestion.
- Review the suggested instructions against the originals. You can edit the suggested text directly in the modal before deciding.
- Click Update instruction to apply the suggestion, or Cancel to discard it with no changes made.
- On success, a confirmation toast appears with a link back to the Configuration tab. The updated instructions take effect immediately for all new evaluations.
Applying an improvement does not automatically update the rule's auto-learn accuracy score. To recalibrate accuracy, re-run Auto-Learning (see 2. Setting Up Agent QA).
Manual rule editing from the Configuration tab's QA Parameters remains available as before and is unaffected by this pipeline.
7.5 Continuous Learning Governance API
To pull a persona's feedback funnel and auto-learn accuracy into an external BI tool or governance report, use the Rule Metrics API. This requires the same elevated persona access as other Agent QA data-pull APIs.
| Property | Value |
|---|---|
| URL | {base_url}/api/agent-qa/rule_metrics |
| HTTP Method | GET |
Query Parameters
| Field | Type | Required | Description |
|---|---|---|---|
persona_id | UUID | Yes | The unique identifier of the AI Employee (persona). |
start_date | string | No | UTC date in YYYY-MM-DD format. Inclusive start of the window. Omit both dates for all-time. |
end_date | string | No | UTC date in YYYY-MM-DD format. Inclusive end of the window. |
Example Request
curl -X GET \
"{base_url}/api/agent-qa/rule_metrics?persona_id=<persona_id>&start_date=2026-07-01&end_date=2026-07-28" \
-H "Authorization: Bearer <token>"Response
A successful response (200 OK) returns one object per active rule:
| Field | Type | Description |
|---|---|---|
persona_id | string | Echoes the requested persona. |
start_date / end_date | string | The resolved date window. |
rules | list[object] | One entry per active rule. See below. |
| Rule Entry Field | Type | Description |
|---|---|---|
rule_name | string | The QA parameter's name. |
auto_learn_accuracy | float or null | 0–1 accuracy from the rule's most recent auto-learn run. null if the rule has no auto-learn history yet — for example, it was added manually or renamed since the last run. |
given | integer | Total feedback submitted for this rule in the window. |
approved | integer | Feedback with status Approved or Processed. |
improved | integer | Feedback with status Processed — i.e., actually incorporated by Continuous Learning. |
Use given → approved → improved as the feedback funnel for a rule, and compare it against auto_learn_accuracy to see whether feedback volume is translating into measurable accuracy gains.