The difference between a chatbot and an agent is what happens after the answer
Every vendor now says "AI agent". Most of them ship a better-worded chatbot. Here is the line that actually separates the two — and how to check which one you are buying.
For fifteen years, a website chatbot was a decision tree wearing a friendly avatar. You drew boxes, connected them with arrows, and hoped the visitor’s question fit one of the paths you had imagined. When it didn’t — and it usually didn’t — the visitor got the same sentence everyone got: “Sorry, I didn’t understand that. Would you like to speak to an agent?”
Language models removed that limitation almost overnight. Suddenly the box could understand anything. So every chatbot became an “AI chatbot”, and then, a season later, every AI chatbot became an “AI agent”.
The words moved faster than the software. Here is the distinction worth caring about.
A chatbot ends at the answer. An agent starts there.
A chatbot’s job finishes when it produces text. It can produce very good text now — fluent, on-brand, contextually aware. But the outcome of a chatbot conversation is always the same shape: the visitor has been told something, and every real-world consequence of that conversation still has to be carried out by a human, or by the visitor themselves.
An agent’s job finishes when something has changed. The appointment exists in the calendar. The lead exists in the CRM with the six fields your sales team actually filters on. The order status came from the order system, not from a paragraph someone wrote about order statuses in 2024.
If the conversation ends and nothing outside the chat window is different, you had a chatbot.
That is the whole test. Everything else — model choice, prompt quality, how natural the writing feels — is a matter of degree. This one is a matter of kind.
What an agent needs that a chatbot doesn’t
Three things, and all three are unglamorous.
Grounded knowledge. A chatbot can be impressive with a good prompt. An agent has to be correct, because it is about to do something on the strength of what it believes. That means retrieval from sources you control and can update — your site, your documents, your catalog — and a model that is instructed to decline rather than improvise when the sources come up empty. Confident wrong answers are annoying in a chatbot and expensive in an agent.
Tools with real permissions. Reading a calendar is a tool. Holding a slot is a tool with consequences. The interesting engineering in agent systems is not the model call — it is deciding which actions the agent may take unsupervised, which need a confirmation step in the conversation, and which get routed to a person. An agent without tools is a chatbot with better vocabulary. An agent with unbounded tools is a liability.
A handover path. The most underrated feature in any agent product is the moment it stops. Refunds outside policy, legal questions, an angry customer, a request that touches money — these should reach a human quickly, with the full conversation attached, not restarted from zero. Agents that never escalate aren’t confident; they’re unsupervised.
The parts that should still be boring
There is a temptation, once the model can do anything, to let it do everything. Resist it in exactly three places:
- Anything irreversible. Cancellations, refunds, deletions, outbound payments. The agent can prepare the action and summarize it; a person presses the button, or the customer confirms explicitly in the conversation.
- Anything about identity. Agents should never collect passwords or card numbers in a chat thread. Hand off to your existing authenticated flow and let the agent pick the conversation back up afterwards.
- Anything the business hasn’t decided yet. If your team cannot state the return policy in one sentence, the agent will invent a version of it. Ambiguity in the business becomes hallucination in the product.
Everything else — the tedious, high-volume, “where is my order”, “do you ship to Austria”, “can I move my appointment to Thursday” traffic that consumes most support hours — is exactly what an agent should absorb.
How to tell what you are actually buying
Vendor demos all look the same. These four questions separate them quickly:
- “Show me the agent doing something, not saying something.” Ask for a booking, a CRM record, a stock lookup — live, in the demo, against a real system.
- “Where did that answer come from?” A serious system can point at the source document or catalog entry behind a specific sentence.
- “What happens when it doesn’t know?” The right answer is a graceful decline plus an escalation, not a plausible paragraph.
- “What can it not do?” A vendor with no answer to this has not thought about permissions, which means you will be the one thinking about them, in production.
The practical version
You don’t need an agent for everything. A well-grounded assistant that only answers questions is already a strong upgrade over a decision tree, and it is a reasonable first deployment — lower risk, faster to trust, easy to measure.
But be honest with yourself about which one you have. If your team is still copying details out of chat transcripts into other systems by hand, the conversation is not finished when the chat ends. It has just been moved onto somebody’s to-do list.
That’s the gap agents close. Not the talking — the part after.
