1. Track Key Performance Indicators
What to measure
What to measure
Essential metrics:
- Involvement Rate: % of conversations with AI participation (target: 80%+)
- Resolution Rate: % resolved without escalation (excellent: 75%+, good: 60-75%)
- CSAT: % of 4-5 star ratings (leading: 80%+, solid: 70-80%)
How to improve
How to improve
- Establish baseline (Analyze → Metrics, last 30 days)
- Set 90-day targets (aim for 10-15pp improvement)
- Weekly reviews (15 min): Check trends, spot issues
- Monthly deep dives (1 hour): Compare month-over-month, segment by channel
- High resolution + low CSAT = accuracy issues
- Low resolution + high “No Answer” = knowledge gaps
- CSAT 80%+ + Resolution 75%+ = healthy performance
2. Identify Poor Performing Topics
What to look for
What to look for
Navigate to Analyze → Topics and review the treemap:
- Large red boxes: High volume + low resolution = maximum impact
- Yellow boxes: Moderate performance, improvement potential
- Green boxes: Success patterns to replicate
How to improve
How to improve
- Click topic to filter conversations
- Review 10-20 conversations to identify root cause:
- Missing knowledge → Add snippets/documentation
- Wrong information → Fix sources
- Poor presentation → Update guidance
- Too complex → Create escalation rules
- Implement fixes and rebuild
- Monitor improvement over 2 weeks
- This week: Top 1-2 topics by impact
- This month: Top 5 topics
- This quarter: All topics below 60% resolution
3. Identify Not Yet Enabled Tickets
What to look for
What to look for
Navigate to Analyze → Conversations, filter to “Not Involved” conversations. Common causes:
- Channels not AI-enabled (80%+ uninvolvement in specific channel)
- Brand/ticket type exclusions (patterns in Zendesk brand or Salesforce case type)
- Historical imports (created before deployment date)
- Outbound messages (agent-initiated, expected)
How to improve
How to improve
- Analyze uninvolved conversation breakdown
- Size expansion opportunity (exclude historical/intentional exclusions)
- Phased rollout:
- Month 1: Enable one excluded segment, monitor quality
- Month 2: Expand if successful
- Month 3: Full rollout across appropriate channels
4. Use Improve Answer to Fix Wrong Information
What to look for
What to look for
Find incorrect responses by filtering:
- 1-2 star ratings with feedback like “wrong,” “incorrect,” “outdated”
- Escalated conversations where agents corrected AI
- Message search for “actually,” “incorrect,” “that’s wrong” (operator messages)
How to improve
How to improve
- Identify root cause:
- Outdated documentation → Update source
- Conflicting sources → Remove duplicates
- Misinterpreted source → Add explicit snippet
- Wrong context → Add conditions
- Fix source (Train → Data Providers or Snippets)
- Rebuild and deploy
- Weekly: Review 10-15 low-rated conversations, fix wrong info
- Monthly: Audit snippets for outdated pricing/features/policies
- Quarterly: Remove deprecated sources, consolidate duplicates
5. Use Improve Answer to Add Missing Information
What to look for
What to look for
Find knowledge gaps via:
- “No Answer” conversations (Analyze → Metrics, Answer Completeness chart)
- Message search for “I don’t have information,” “I’m not sure,” “I couldn’t find”
- Unresolved + Fully Autonomous conversations
How to improve
How to improve
Quick fix: Create snippet (click AI message → Add Snippet → write ideal answer)Long-term: Create comprehensive documentation for topic clustersChoose source type:
- Snippets: Single Q&A, quick gaps, policy clarifications
- PDFs: Product manuals, process docs
- Webpage crawls: Help center, living documentation
- Tables: Pricing, specs, comparisons
- Database integrations: Order status, account info
- New product launch: Document before release, test AI responses
- Seasonal prep: Holiday shipping, tax season FAQs
- Track recurring unanswered questions weekly, prioritize by frequency
6. Foster a QA Culture Among Your Team
What to set up
What to set up
Label structure (Settings → Labels):
- Workflow: “QA: Needs Review,” “QA: Reviewed,” “Review: [Name]”
- Findings: “Knowledge Gap - [Topic],” “QA: Excellent Example,” “Wrong Information”
- Monday: Team lead assigns 5 conversations/person with labels
- During week: Team reviews, applies finding labels, documents in shared doc
- Friday: Team lead reviews findings, creates updates, deploys, removes temporary labels
How to sustain
How to sustain
Start small:
- Week 1-2: Review 5 conversations, document only
- Week 3-4: Add labels, create 1-2 snippets
- Month 2+: Full workflow with regular deployments
- Shared doc for weekly findings and actions
- 15-30 min weekly sync: metrics review, share findings, plan actions, celebrate wins
- Rotate review topics monthly for cross-training
- Set boundaries: 10 reviews/week max per person
- Celebrate excellent responses publicly
7. Review DSAT Conversations
What to look for
What to look for
Filter to 1-2 star ratings (Analyze → Conversations or Metrics → click rating chart). Check:
- Customer feedback text for patterns: “didn’t answer,” “wrong,” “too long,” “misunderstood”
- User context sidebar: device (mobile/desktop), page visited, previous conversations
- AI message sources (Improve Answer sidebar): zero sources, wrong sources, or poor presentation
- Knowledge gap (40%) → Missing information
- Wrong information (20%) → Incorrect sources
- Misunderstood question (15%) → Guidance issues
- Poor presentation (15%) → Formatting/tone
- Inherent complexity (10%) → Needs human
- Unrealistic expectations (5%) → No action needed
How to improve
How to improve
Process 10-20 DSAT conversations weekly:
- Read full thread, identify root cause
- Batch by category (knowledge gaps, wrong info, presentation)
- Create action plan with priorities
- Implement fixes (snippets, source updates, guidance changes)
- Track DSAT reduction by topic month-over-month
Monthly Optimization Routine
Week 1: Review & Prioritize
Week 1: Review & Prioritize
- Monday (30 min): Review metrics dashboard, compare to previous month
- Wednesday (45 min): Review Topics treemap, prioritize top 3-5 by impact
- Friday (30 min): Export metrics, share with team
Week 2: Deep Dive & Assign
Week 2: Deep Dive & Assign
- Monday (45 min): Review 15-20 conversations in priority topic, create action plan
- Wednesday (1 hour): Assign QA reviews to team (20-25 conversations)
- Friday (30 min): Review team findings, prioritize fixes
Week 3: Create & Deploy
Week 3: Create & Deploy
- Mon-Wed (2-3 hours): Create snippets, fix sources, update guidance
- Thursday (1 hour): Review changes, build new profile
- Friday (30 min): Deploy, announce improvements
Week 4: Validate & Plan
Week 4: Validate & Plan
- Monday (30 min): Check deployed changes, look for issues
- Wednesday (1 hour): Compare metrics to Week 1, measure impact
- Friday (1 hour): Team retrospective, set next month goals, celebrate wins
Measuring Success
Track these indicators:- Lagging (results): CSAT ↑, Resolution ↑, DSAT ↓, Escalations ↓
- Leading (activities): Conversations reviewed, snippets created, deployments, QA participation
- Metrics plateau → Focus on guidance refinement vs knowledge additions
- Metrics regress → Review recent changes, consider rollback
- Team engagement drops → Reduce review quota, increase recognition
Next Steps
Metrics Dashboard
Track your performance indicators and measure improvement over time
Topics Analysis
Identify poor-performing topics and prioritize optimization efforts
Improve Answers
Learn how to fix wrong information and add missing knowledge
Labels for QA
Set up label workflows for systematic quality assurance