Short answer
A support assistant earns its place when it resolves repetitive questions completely, from your own documentation and order data, and hands everything else to a person immediately with the transcript attached. Scope it to the questions that make up the bulk of your volume, ground it in your content rather than general knowledge, say plainly that it is software, and measure conversations fully resolved without a human — not conversations started.
Deflection is the only metric that matters
Vendor dashboards are full of numbers that look like success and are not: conversations handled, messages sent, questions answered. All of them go up when the assistant is doing badly, because a customer who does not get an answer asks again.
The number worth tracking is the share of conversations that ended with the customer's problem solved and no human involved. That figure is harder to compute and impossible to inflate. Track alongside it the share that escalated, and — the one nobody instruments — the share where the customer gave up and left without either.
If abandonment rises after launch while ticket volume falls, you have not deflected anything. You have made it harder for customers to reach you, and the effect will surface later as churn rather than as tickets.
Record all three numbers before launch, not after. Without a baseline taken while the process is still manual, any change you observe afterwards is an opinion — the failure mode we set out in measuring automation ROI honestly.
Scoping it: three questions
Before choosing any tool, answer these about your actual support volume.
- Which questions make up the bulk of it? Pull three months of tickets and group them. In most companies a handful of question types dominate, and that handful is the entire initial scope.
- Which of those have a definitive, written answer? If the answer depends on judgment or on information that lives in someone's head, it is not a candidate yet. Write the answer down first; that alone reduces tickets.
- Which need data lookup rather than knowledge? Order status, delivery date, account state. These are the highest-value cases because the answer is unambiguous and the customer wanted it instantly — but they need a real integration, not a document.
Scope to that list. Resist the temptation to cover everything at launch: a narrow assistant that is right is trusted, and a broad one that is often wrong gets bypassed permanently after two bad experiences.
Grounding: answering from your content only
A general-purpose model asked about your refund policy will produce something that sounds like a refund policy. That is the core risk, and the mitigation is architectural rather than a matter of prompt wording.
The assistant should retrieve from your published documentation, your policies and your live systems, and answer from what it retrieved — with the source visible to the customer where possible. When nothing relevant is retrieved, the correct behavior is to say so and offer a person, not to improvise. That mechanism, and its failure modes, is the subject of making an assistant answer from your own files.
One consequence people do not expect: this makes the quality of your documentation the binding constraint. Teams frequently discover during a chatbot project that their own written answers are contradictory or out of date. Fixing that is not a detour — it is most of the value.
The escalation path everyone underbuilds
Ask any customer what they hate about support bots and you will get one answer: they could not get to a human. This is the feature that decides how the whole deployment is remembered.
- Always available. A visible way to reach a person at every point in the conversation, not only after the bot has failed three times.
- No repetition. The transcript travels with the customer. Being asked to explain the problem again is the moment goodwill is lost.
- Honest about timing. If nobody is available until Monday, say so and take a message. Silence after an escalation is worse than the escalation.
- Triggered automatically on frustration. Repeated rephrasing, an explicit request for a human, or any hint of anger should route out immediately without waiting for a failure threshold.
Where chatbots actively hurt
There are situations where deploying one costs more than it saves, and they are predictable.
Complaints and anything emotionally loaded should reach a person on the first message. Anything with financial or legal consequence — cancellations, disputes, contractual questions — needs a human, both for accuracy and because the customer will not accept the answer otherwise. And low-volume, high-value support, where every customer is known by name, gains almost nothing and risks looking impersonal.
The pattern is consistent: the more the customer's emotional state matters to the outcome, the worse the trade. That is not a limitation of the current models so much as a property of the situation.
A rollout that does not embarrass you
- Run it internally first, answering your own team's questions. They will find the failure modes without costing you a customer.
- Launch on one narrow question type, clearly labeled, with the escalation path prominent.
- Read the transcripts weekly, personally, for the first month. Dashboards hide the specific bad conversation that tells you what is wrong.
- Expand one question type at a time, and only after the previous one is resolving reliably.
- Keep a kill switch that a non-technical person can operate. If it starts giving bad answers on a Saturday, someone must be able to turn it off.
Whether this justifies an agent architecture or a much simpler retrieval-plus-rules design is worth deciding deliberately rather than by default — see what AI agents actually do.
Frequently asked questions
Will a chatbot reduce our support headcount?
That is the wrong thing to promise and usually the wrong thing to aim for. What a well-scoped assistant reliably does is absorb the repetitive questions that arrive at all hours — where is my order, how do I reset this, what are your terms — so the same team spends its time on the cases that need a person. Companies that plan headcount reductions before measuring actual deflection tend to end up with a worse service and the same costs.
Should the bot say it is a bot?
Yes, and not only for legal reasons. Customers work out within two exchanges that they are talking to software, and discovering it after being misled converts mild annoyance into a complaint. Stating it plainly at the start costs nothing and sets expectations correctly — people ask simpler, better-formed questions when they know what they are talking to.
What is the single most important feature?
The escalation path. Not the model, not the interface, not the number of languages. A customer must be able to reach a person quickly, without repeating themselves, and the transcript must arrive with them. A bot that cannot resolve and cannot escalate is a queue with extra steps, and it is the configuration that generates the complaints people associate with the whole category.
How do we stop it inventing answers?
Ground it in your own content and constrain it to that. An assistant answering from your published documentation, order data and policies has a much narrower space in which to be wrong than one drawing on general knowledge. Then instrument it: log every conversation, sample them weekly, and make "I do not know, let me get someone" an acceptable and frequent answer rather than a failure state.
If you want an assistant scoped to what your customers actually ask, grounded in your own content, we build those. Book a call and bring three months of tickets.
