engineering notes
GREEN bias: how development environments falsify AI-agent quality
Why testing AI agents yields green reports while the agent fails in production: the built-in GREEN bias of development environments, and how to bypass it with honest evaluation.
Executive summary. Green tests for an AI agent often mean not “the agent works” but “it was checked under conditions where it couldn’t fail”. Development environments and the people writing tests carry a built-in bias toward a green result — they unconsciously help the system. As a result you can get an excellent quality report and a poor product at the same time. The fix isn’t more tests but an independent check that is interested in failure.
When an agent passes its tests, the quality question seems closed. In reality a green report says more about the test than about the agent. Testing AI agents has a systematic bias toward success — call it GREEN bias — and until you see it, you mistake a convenient picture for a measurement.
A test that plays along with the agent checks the author’s expectations, not the system’s strength.
Hypothesis: environments and testers unconsciously pull the result toward green
A test is written by someone who knows how the system works — and unconsciously phrases the input so the agent succeeds: clear, contradiction-free, single-branch. Modern agent environments amplify this: they simplify the scenario, fix typos, complete partial input, pick the most likely interpretation of intent. Each such “convenience” nudges the run toward green. In sum, the system subconsciously helps itself pass, and the green report stops proving anything.
Problem: a green report and a working agent are different things
Because of GREEN bias it’s easy to get a high pass rate and low production quality. The test checked knowledge of facts under hothouse conditions; reality checks behavior in chaos. The danger isn’t the error itself but the false signal: management sees a green report, reads it as “done”, and ships the agent into a customer-facing loop. From there the edge is found by the customer, not the team — and that’s no longer a CI line but a ruined conversation and reputational risk.
Adoption grows by an order of magnitude, yet nearly half of projects never reach an outcome. The gap between a demo and a working system is closed by architecture, not by the model.
The gap shows in adoption figures: agentic AI is deployed ever more widely, but only about half reaches a working result. That is precisely the gap between “passed the demo” and “works with people” — and it closes not by changing the model but by honest checking.
Why the usual approaches don’t work
The first reflex — “add more tests” — doesn’t help, because new tests are written by the same people in the same environment, carrying the same bias. More green under convenient conditions yields zero red under inconvenient ones. The second approach — “run once and freeze” — ignores non-determinism: the same input gives an LLM different answers, so a single green run guarantees nothing; you need statistics, not a checkmark. The third — “let the developer test it” — runs into a conflict of interest: the author checks their expectations, not the strength. The root is one: the check is run by a party interested in green.
Engineering model: how to bypass the bias
For evaluation to be honest you need a loop interested in red, and conditions where the bias doesn’t work. In practice that’s three things. First, an independent attacking agent whose job is not to help but to break: confuse, derail, catch a contradiction, force an invented answer. Second, non-determinism handled explicitly: every scenario is run many times, and the metric is the share of successful outcomes and the rate of confident errors, not a single “passed”. Third, input taken from real dialogs, not synthetics: people stumble in ways a developer doesn’t imagine, and it’s exactly those stumbles that break the agent. This is the move from a demo to a working system.
Most pilots «answer» but produce no impact — because an interface is built, not a process with state, contracts and control.
The same bias shows in generative-AI pilots: most “answer”, but only a few drive rapid revenue. Between “answers nicely in a demo” and “moves a business metric” lies exactly the check that GREEN bias hides.
Practical takeaway for the business
Don’t accept a green report as proof. Ask who wrote the tests and whether anyone was interested in failure. If the same team that built it did the checking, you have no independent assessment — you have a self-check.
What to delegate. Independent agent testing is a separate function, like a security review. The minimum to start: an attacking agent plus a run on a corpus of your real dialogs, with a metric across many attempts. It’s cheap relative to the cost of an agent that confidently erred in front of a customer.
What not to do. Don’t ship an agent on a green report from its own authors. Don’t confuse green CI with readiness: the first is hygiene, the second is about whether a real person reaches the result. And don’t pile on more tests instead of adding an interest in red to the loop.
Check your agent for the bias — .
Open questions
Can GREEN bias be removed entirely — no, but it can be made visible and offset by an independent loop. Where’s the line between “environment convenience” and “falsification” — where the environment starts fixing input that no one will fix in production. Who’s accountable for “the report is honest” — that’s a management question, settled before launch, not after the first customer incident.
If you have green tests but no confidence in the agent with real people, that’s the symptom of GREEN bias. — let’s see what your tests aren’t checking yet.