engineering notes
Why a chatbot is not an AI architecture
How a corporate chatbot differs from an AI system: state, contracts, human control — and why this decides money and risk.
In brief for executives. “We need a chatbot” is almost always not a problem statement but the name of the first thing that came to mind. A chatbot answers; a system does the work. The difference is not cosmetic: it determines whether money and accountability can be entrusted to the process. A request for a chatbot is worth reframing into the question “which process do we want to remove from people” — otherwise the budget goes into a system that answers nicely and executes nothing.
The request “make us a chatbot” comes often and sounds specific. But behind it usually stands not a need for a chat window but a need to remove some work from people. These two are confused, and because of that an interface is obtained where a system was needed.
A chatbot answers. A system does the work.
Hypothesis: a chatbot is a stateless model call, a system is a process
A chatbot at its core is a single model call without state: a question came in — an answer came back. It does not remember what it did a minute ago, changes nothing in your systems, and is accountable for nothing. An AI system is a process: it has state, input/output contracts, tool access, failure handling and points where a decision is handed to a human. A chatbot speaks. A system acts. This is not “an advanced chatbot” — it is a different class of object.
Problem: “chatbot” substitutes for the problem statement
When the task sounds like “make a chatbot”, a decision is quietly fixed before the problem is stated. Then people discuss which model to take and how to write the prompt, not which process is being automated and how to measure the effect. A couple of months later there is a bot that politely answers questions, and no answer to whether what it was all started for got faster. Formally the project is done; in essence it isn’t.
Why the usual approaches don’t work
Trying to “grow” a chatbot into a system by adding prompts doesn’t work, because what’s missing is not answer quality but architecture. A chatbot has no state — so it cannot run a multi-step process: each answer exists on its own. It has no contracts — so its answer cannot be reliably passed to another system: free text is not an interface. It has no failure handling — so on an external-service failure it will either fabricate an answer or just apologize, but not escalate. It has no handoff point — so it will make an expensive decision as easily as a cheap one. Each of these is not a prompt setting but a separate architectural layer absent from a chatbot by definition.
Most pilots «answer» but produce no impact — because an interface is built, not a process with state, contracts and control.
Engineering model: what separates a system from a bot
To turn “answers” into “does the work”, four things are needed that a chatbot lacks.
Process state. The task lives longer than one message: what’s done, what remains, which data arrived. Only thus is a multi-step process possible.
Contracts at the seams. Input and output are typed, with an explicit “unsure / couldn’t” flag. Then the result can be passed to a CRM, to another system, to the next step — and checked.
Failure handling. An external service didn’t respond, data is contradictory, confidence is low — these are foreseen process branches, not “the model will think of something”.
Handoff to a human as a step. Where the price of an error is high, the decision routinely goes to a human with the gathered context. This is a designed boundary, not a sign of failure.
Practical takeaway for business
Reframe the request. Instead of “we need a chatbot” — “which repeated process do we want to remove from people and how will we measure the effect”. If there’s an answer — you need a system, and a chat window in it is at best one of the interfaces. If there’s no answer — the project isn’t ready, and no bot will close that.
When a chatbot is enough. If the task is to answer typical questions with no consequences (reference, product navigation) and the error cost is near zero, a bot is appropriate and there’s no need to complicate. The criterion is simple: if the answer affects nothing in your systems and is accountable for nothing — a bot; if it affects or is accountable — a system. This criterion is stated in terms of money and risk, not technology, so the decision here is a management one.
The price of the confusion is not the price of the bot. The bot itself is cheap. What’s expensive is that under the guise of a solved task the real one stays unsolved, plus the company’s hardened conclusion that “AI didn’t work for us”.
Failure is systemic and predictable: not the model, but the absence of architecture (state, contracts, failure handling, handoff to humans).
Apply this to your processes — .
Open questions
Where exactly the “a bot is enough” line runs depends on the error cost in the specific process and is decided deliberately, not by default. Whether a bot can be evolved into a system — sometimes yes, but more often it is a rewrite: state and contract architecture cannot be “added on” over a stateless solution without rework. Who is accountable for a decision the system made without a human is a management question decided before launch.
If behind “we need a chatbot” stands a process you want to remove from people — name that process, and it will be clear whether a system is needed or a bot is enough. — we’ll decompose the request into process, error cost and the required class of solution.