Everyone who tried a chatbot between 2016 and 2021 has the same memory: a decision tree that could not understand a sentence, endlessly offering four buttons, none of which matched the question. Those systems earned their reputation. The scepticism they created is rational and still widespread.
What changed is not incremental. It is the difference between matching keywords and understanding language. Worth understanding precisely, because the word "chatbot" is still used to sell both.
Key takeaways
- Rule-based bots match keywords against a script. They break on any phrasing the author did not anticipate.
- LLM agents interpret intent, so paraphrase, typos and mixed languages stop being failure modes.
- The real differentiator is grounding: an agent that cites your catalogue beats one that improvises.
- Handoff quality matters more than resolution rate. Knowing when to stop is the hard part.
Why rule-based bots failed
A rule-based bot is a flowchart. The author lists the phrases they expect, maps each to a response, and everything unanticipated hits a fallback: "Sorry, I didn't understand that."
This fails because language is effectively infinite. "Where's my order", "has my parcel shipped", "still waiting on my delivery" and "order #4471 hasn't arrived" are one intent in four shapes. A rule-based system needs every shape written out in advance. Miss one, and the customer hits a wall.
Teams responded by writing more rules. The trees grew to hundreds of nodes, became impossible to maintain, and still broke. The technology could not represent the problem.
What an LLM agent does differently
An LLM-based agent does not match strings. It interprets meaning, then decides what to do. In practice that produces four concrete differences:
- Paraphrase stops mattering. All four order-status phrasings resolve to the same intent without anyone enumerating them.
- Context persists. "What about the blue one?" resolves correctly because the agent retains the preceding turns.
- Language is not a separate build. One agent handles every language it is configured for, in a consistent tone.
- Messy input survives. Typos, voice-note transcripts, and switching languages mid-sentence are handled.
Rule-based bots ask the customer to phrase things the way the system expects. Agents adapt to how the customer already speaks.
Where agents still fail
An agent that is not constrained will confidently invent things. Not maliciously, and not obviously. It produces plausible, well-written, entirely fabricated answers, which is far more damaging than an honest "I do not know" because customers believe it.
Three constraints separate systems that work from systems that embarrass you:
- Grounding. Answers must be drawn from your catalogue, policies and docs, not from the model's general knowledge. If the source does not cover it, the agent should say so.
- Scope. You define the topics it may speak on. Anything else escalates. This is what stops an agent from improvising a refund policy.
- Confidence thresholds. Below a certain certainty, hand off. An agent that knows what it does not know is the entire ballgame.
The question to ask a vendor. Not "how accurate is it?" but "what happens when it is unsure?" A vendor without a crisp answer is selling you a system that will guess in front of your customers.
How to tell them apart when buying
Vendors use the same vocabulary regardless of what is underneath. Four questions separate them quickly:
| Ask this | Rule-based answer | Agent answer |
|---|---|---|
| How do I add a new question? | Add an intent and training phrases | Add the source content |
| What if a customer phrases it oddly? | Add it to the training set | Already handled |
| How do I add a language? | Rebuild the tree per language | Enable it |
| What happens when unsure? | Falls back to a menu | Escalates with a summary |
The tell is maintenance. If a demo impresses you but adding a question means editing a flowchart, you are buying the old thing with new branding.
Frequently asked questions
Will an AI agent replace my support team?
No, and the framing is wrong. It absorbs the repetitive share of volume so your team handles the conversations that genuinely need judgement. Most teams keep headcount and stop drowning.
Can it invent a refund policy?
Only if you let it. With scope control and grounding, policy questions outside its sources escalate rather than being answered.
Do customers mind talking to an AI?
Satisfaction tracks answer quality and handoff speed far more than whether the responder is human. Slow humans score worse than fast, accurate agents.