Conversation review and quality assurance — keeping the chatbot honest
How to monitor what Voxinly is actually saying to your customers, catch bad answers early, and retrain via knowledge-base updates. The weekly 10-minute review that prevents drift.
Voxinly is only as good as the knowledge you've given it and the patterns it's learned about your business. Even a well-configured chatbot will say a wrong thing occasionally — quote the wrong price for an unusual job, confuse two different services, suggest a postcode is in your patch when it isn't. Quality review catches these before they cost you a booking.
The good news: most bad answers come from one of three fixable sources — a missing knowledge-base entry, an outdated price, or a poorly-phrased rule. Spot the pattern in review, fix the source, and the same mistake won't happen again.
Step 1 — Where to find conversations
Open /admin/customers
Every conversation, ever, lives here. Sort by date, filter by status (booked, abandoned, declined), filter by quality flag, search by keyword or postcode.
Step 2 — The quality flags
Voxinly auto-flags conversations that show low-quality signals. Five flags to watch:
| Flag | What it means | Priority |
|---|---|---|
| Bot apologised | The chatbot said 'I'm not sure' or similar — knowledge gap | High |
| Customer confused | Customer asked the same thing twice or rephrased | High |
| Abandoned mid-quote | Customer left after pricing was shared | Medium |
| Negative sentiment | Customer expressed frustration or sarcasm | Medium |
| Unanswered question | Bot replied to a different question than asked | High |
Filter to flagged conversations
In /admin/customers, click the 'Quality flags' filter. You'll see only the conversations that triggered one of these signals. Most weeks you'll have 5-15 to review.
Step 3 — The weekly review ritual
Block 10 minutes every Friday afternoon. Open the flagged conversations one by one. For each, ask three questions:
- What did the customer actually want?
- What did the bot say (or not say) wrong?
- What knowledge-base entry would have fixed it?
Note the fix and move on. Don't try to perfect every conversation — go for the patterns. If three customers asked something the bot couldn't answer this week, that's the knowledge gap to fix first.
Step 4 — Fixing via the knowledge base
Add the missing knowledge
Open /admin/knowledge → Add document. Title the entry by the customer question ('How long does a boiler service take?'). Write the answer in plain English. Save. The bot incorporates it on the next conversation, usually within a minute.
Step 5 — Fixing via quote rules
If the bot quoted the wrong price, check /admin/chatbot → Quote rules. The wrong price usually traces to an outdated number or a missing edge case (e.g. a Worcester Bosch boiler service vs Vaillant — same price for you, but the bot couldn't tell). Add the edge case as a new rule and the bot won't make the same mistake.
Step 6 — Logging fixes
Voxinly keeps a quality-fix log in /admin/quality. Every time you make a knowledge-base or quote-rule change in response to a flagged conversation, log it briefly ('Added boiler-service duration entry'). After eight weeks you'll have a record of every improvement — useful for seeing patterns.
Step 7 — The first 8 weeks vs after
In the first 8 weeks the chatbot is still learning your business edge cases. Review weekly. From week 9 onwards, flag rate drops sharply — most firms can move to monthly review after that. The bot won't get worse, but customer questions will keep evolving (new pricing, new services, seasonal demand) so don't drop reviews entirely.
Step 8 — Customer-reported issues
Sometimes a customer will email you saying 'your chatbot quoted £200 but the actual price was £350 — what gives?'. Open the conversation, see what the bot said, identify which knowledge entry or quote rule misfired, and fix the source. Reply to the customer with the correct figure and (importantly) tell them the fix has been made. They'll usually still book.
FAQ
Can I edit a conversation transcript if the bot said something wrong?
No — transcripts are audit-only. You can flag a message as 'needs review' for your records, but you can't rewrite history. The fix is forward-looking, via knowledge updates.
What if the bot is consistently quoting low?
Check the knowledge base for an outdated price entry. Quote rules should reflect current pricing — update the source, the bot follows.
Should I review the good conversations too?
Quick scan only. If you spot something the bot did particularly well, share with your team — useful training material. Don't dwell.
What if I disagree with a quality flag?
Click 'mark as good' on the conversation. Voxinly learns from your overrides over time. After enough overrides, that pattern stops triggering the flag.