Feedback and Continuous Learning
Continuous Learning improves evaluation accuracy over time through reviewer feedback. Unlike Auto-Learning (which uses a batch of golden data during initial setup), Continuous Learning incorporates corrections incrementally as your team reviews evaluations.
7.1 Submitting Feedback
The feedback UI in the evaluation details drawer is currently under development. The backend API fully supports all feedback operations. The workflow described below reflects the intended user experience once the UI is enabled.
Permission required: CAN_VIEW_PERSONA (Admin, Manager, or Builder)
- In the Audit tab, click View Details on an interaction
- In the Scorecard, find the parameter you want to comment on
- Click the thumbs down icon (disagree with evaluation) or thumbs up icon (agree)
- Enter a feedback comment explaining your assessment
- Click Submit
What Makes Feedback Effective
Good example: "The agent did say 'Team Portugal' at 0:15, which should count as company identification. Ensure that any variation of the company name in the opening counts toward this rule."
Tips: Be specific about what happened in the call (reference timestamps if possible). Explain what the correct evaluation should be and why. Describe the general principle, not just the one-off correction. Avoid vague comments like "This is wrong."
Rules for feedback management:
- Only the feedback creator can update or delete their own feedback
- Updating feedback resets its status to unprocessed, requiring re-moderation
7.2 Moderating Feedback
Permission required: CAN_EDIT_PERSONA (Admin or Builder)
- Navigate to the Audit tab and open the feedback section
- Review pending (unprocessed) feedback items
- For each item: read the original evaluation result and rationale, read the feedback comment, check the transcript for context
- Click Accept to approve, or Reject to ignore
Feedback Lifecycle
| Status | Description |
|---|---|
| Unprocessed | Submitted, awaiting moderation |
| Approved | Accepted by a moderator. Eligible for the Continuous Learning pipeline. |
| Processed | Already consumed by the Continuous Learning pipeline. Will not be reused. |
| Ignored | Rejected during moderation. Visible but not used for learning. |
7.3 Generating Improvement Suggestions
The improvement suggestion workflow described here is a planned feature. Currently, parameter refinement is done manually or by re-running Auto-Learning.
Once enough approved feedback accumulates for a parameter, the system can generate improvement suggestions:
- The system analyzes all approved feedback for the target rule
- Fetches the transcript and evaluation data for each associated interaction
- Generates updated instructions:
| Output | Description |
|---|---|
| Updated rule text | New pass/fail instruction incorporating feedback corrections |
| Updated N/A text | New not-applicable instruction |
| Rationale | Explanation of why changes were suggested |
| Confidence | High, medium, or low |
7.4 Applying Improvements
This section describes planned functionality.
- Review the suggested instructions side-by-side with the originals
- Verify the rationale aligns with your QA standards
- Click Update parameter to apply, or Cancel to dismiss
- Updated instructions take effect immediately for all new evaluations
Applying improvements does not automatically update accuracy scores. To recalibrate accuracy, re-run Auto-Learning.