User lying as a normal operating mode

Executive summary. Everyone talks about language-model hallucination: the model confidently states untruths. Far fewer acknowledge the other half of the same problem — the user is unreliable too. People routinely tell the agent the wrong budget, the wrong dates, false constraints and a substituted goal, and almost always not out of malice. An agent that blindly trusts the input builds a flawless solution on false premises — and errs all the more confidently the cleaner its logic. Reliability comes not from believing the customer’s data but from designing for knowingly unreliable input. That changes the requirements for the system and for its acceptance.

In discussions of dialog-AI reliability, almost all attention goes to model hallucination: the LLM confidently says things that aren’t so. That’s a real problem, but it eclipses a second one that’s no less important. A dialog has two participants, and it’s not only the agent that’s unreliable. The human is unreliable too. They report the wrong figures, mix up dates, name false constraints and substitute the goal along the way — and they do it constantly, as a normal mode, not as a rare exception.

Hypothesis: unreliable user input is the norm, not a glitch

The standard dialog-agent architecture silently assumes the user’s data is true: said a budget, so that’s what they have; named a date, so that’s when they’ll go. We claim this assumption is false by default. Unreliable input is not a rare anomaly you can write off as a “difficult client” but a normal operating mode of any dialog system with live people. And, importantly, it’s a mirror of model hallucination: the industry has learned to design defenses against the LLM lying but barely designs defenses against the human lying. An agent’s reliability is determined by whether it survives unreliability on both sides, not just one.

Problem: people distort data constantly, and almost always without malice

It’s worth stripping the moral coloring right away. “Lying” here is not about deception and ill intent; it’s an engineering term for unreliable input. People distort data for entirely understandable reasons, and almost never to do harm.

They name a lowered budget because they’re bargaining or embarrassed by the real figure. They say “flexible dates” while actually tied to a vacation window — they just haven’t yet realized the rigidity themselves. They state a constraint (“direct flight only”) that on inspection turns out to be a preference they’ll drop for the price. They frame a goal (“looking for a gift”) while actually choosing for themselves. They answer “yes, all good” to finish faster, not because it’s good. In none of these is there malice — there is ordinary human imprecision, bargaining, self-deception and a wish not to overcomplicate.

For the agent the result is the same regardless of motive: the input is unreliable. And a linear agent, trusting it, builds a perfectly logical solution on a false foundation. The cleaner its reasoning, the further the result drifts from the real task. This is the same mechanic as a RAG system’s confident error producing a plausible answer on corrupted data — only the source of distortion is not the knowledge base but the interlocutor.

Why the usual approaches don’t work

The first familiar approach is fact validation: check that the budget is a number, the date is in the future, the field is filled. This catches typos and misses the point. A lowered budget passes any fact validation: it’s a correct number, just not true. “Flexible dates” is a valid answer, just unreliable. Formal field checking is blind to data that is internally consistent yet doesn’t match the person’s real task.

The second approach is to train the user to “answer correctly”: introduce strict forms, mandatory fields, “are you sure?” reprompts. This offloads an engineering problem onto the customer and doesn’t solve it: the person isn’t lying on purpose, so they’ll confirm the unreliable answer just as confidently. A rigid form merely disguises distortion as valid input — and scares off those who haven’t decided yet. This is the same hardcoded script that breaks against a live human: we’re trying to force non-deterministic input into a deterministic frame.

The third approach is to hope for the model’s “understanding”: a smart LLM will notice the inconsistency itself. Sometimes it does, but you can’t rely on it systematically. By default the model is obliging and trusting: it takes what’s said as truth and tries to help on those premises, because it’s trained to be helpful, not suspicious. Without an explicit engineering instruction to check motivation, the model faithfully amplifies the distortion — building an ever better answer on the same false basis.

Engineering model: validate motivation, not just facts

The working model shifts focus from checking facts to checking motivation. The agent’s question is not “are the data formally correct” but “do they match the person’s real task”. The loop rests on three things.

First — treat input as a hypothesis, not as truth. Every stated constraint, figure and goal the agent holds as a version requiring confirmation by behavior, not as a fact. This changes the internal model of the dialog: the agent works not with “the customer’s data” but with a picture of their task that it builds out and refines along the way — which is its primary working resource.

Second — surface the motive behind the data, not the data itself. Instead of “please confirm the budget”, the agent gently tests the rigidity of the constraint: what’s critical and what’s a preference the person will drop. A lowered budget surfaces not through an interrogation about the sum but through the reaction to slightly higher options: if the person looks, the figure was a bargaining move, not a ceiling. This is cheap in turns and extracts the real frame of the task, which fact validation simply can’t see.

Third — catch contradictions along the way, not at the input. Unreliability more often shows up not at the moment of the answer but later: “flexible dates” collide with “needs to be before the 15th”, “economy only” with interest in something pricier. An agent that holds the context of the whole dialog and reconciles new messages with old ones notices these discrepancies and reasks to the point — instead of filing the contradiction into the customer profile as two equally valid facts.

Crucially: this model doesn’t make the agent suspicious of the person. It makes it resilient to normal human imprecision — exactly as hallucination defenses make the model itself resilient. Unreliability on both sides is part of the normal noise of dialog, the very thing a reliable agent is designed for.

Practical takeaway for the business

Recognize unreliable input as a design requirement. If the spec says the agent “receives budget, dates and goal from the user”, the spec is already wrong — it describes a demo, not production. The right framing: the agent works with knowingly unreliable input and checks motivation, not facts. This requirement, like hallucination defense, must be in the specification, not appear after the fact once complaints come in.

What to delegate. Build into acceptance the scenarios with distorted input: a lowered budget, falsely flexible dates, a substituted goal, a hasty “yes”. Check against a corpus of real dialogs — that’s where you see how and where people actually distort, rather than how a developer imagines it.

What not to do. Don’t fix the problem by tightening forms and mandatory fields — that offloads engineering debt onto the customer and scares off revenue. Don’t treat fact validation as a defense against unreliability: what’s formally correct and substantively false passes straight through it. And don’t confuse resilience to imprecision with distrust of the person — the goal isn’t to catch the customer out but to bring them to a result on an honestly understood task.

Apply this to your agent — talk to an engineer → .

Open questions

Where the line runs between checking motivation and being intrusive — an over-meticulous agent starts irritating those who did tell the truth, and calibrating this edge rests on live traffic, not intuition. How to measure “the agent is resilient to unreliable input” in acceptance — this is harder than fact-checking and requires a judge by dialog outcome, not by the fact of an answer. How deeply to dig into motive in a short service contact versus a long sale — depends on the cost of error in the specific process and is settled before launch, not in production.

If your agent builds flawless answers that in practice miss, check not the model but the input — most likely it trusts the customer’s data as truth. Discuss your case → — let’s see where unreliable input is leading your agent away from the real task.