engineering notes
Orchestrating AI agents in business processes
What AI-agent orchestration is: how to connect agents, tools and people into a managed business process with cost control.
In brief for executives. The manageability and cost of an AI process are set not by the choice of model but by the orchestration layer — how agents, tools and people are connected into one process. The value of AI for the company is largely that it removes coordination: task routing, data reconciliation, status tracking, on which a significant share of the work week is spent now. Orchestration is what turns a set of agents into a process predictable in cost and control.
“Orchestration” sounds technical, but behind it is a quite managerial question: who, and by what rules, connects the work of several agents, tools and people into one process — and can that process be managed. Let’s go through it.
Value is in the orchestration. The model is a swappable primitive.
Hypothesis: value is in orchestration, the model is a swappable primitive
In a working AI system the model is a replaceable component on a specific step. What creates value and sets cost lies around it: step routing, state, failure handling, handoff to a human. That is orchestration. So “which model to take” is not the project’s main question; the main one is how the process over the models is built.
Almost half the week goes not to the work itself but to coordinating it and finding context. Internal knowledge search cuts that by up to a third — exactly where the AI layer plugs in.
Nearly half a worker’s week goes not to the work itself but to coordinating it and finding context. That is exactly the layer orchestration plugs into — it removes the coordination load, not “answers questions”.
Problem: “orchestrator” is reduced to a loop of model calls
A frequent picture: the orchestrator is a loop that calls the model, passes its answer to the next call, and so round and round. Such an “orchestrator” has no state, no routing by result, no failure handling. It works in a demo and becomes unmanageable on a real flow: cost is unpredictable, behaviour is not reproducible, debugging is impossible.
Why the usual approaches don’t work
A loop of calls without state does not scale to many-step processes: each answer exists on its own, the process doesn’t remember what’s done.
Routing “by the model freely reasoning on every step” makes the system unpredictable in cost and behaviour: the same input may take different paths.
A single strong model “for everything” is unjustifiably expensive: most steps (classification, extraction, short rephrasings) don’t need a strong model, yet you pay for it everywhere.
Engineering model: the orchestration layer
Routing by step result. The next step is chosen by the previous step’s typed output, not by free reasoning. The route is reproducible and testable.
Process state. A task lives longer than one call: what’s done, what’s left, which data arrived. This is the basis of multi-step processes and idempotency.
Model per step. The model decision is a function of the step: risk, required quality, latency. Routine steps on a cheap fast model, expensive decisions on a strong one. This removes the main share of spend without quality loss where it wasn’t required.
Handoff to a human as a process step. Expensive and uncertain decisions routinely go to a human with the gathered context — a designed boundary, not a failure.
Observability by step. Cost, latency and error rate are collected per step. Without it “the system is expensive” cannot be turned into “step X is expensive because of Y”.
Per IDC, of 33 launched pilots only about 4 reach production. The cause of failure is not technology — it is the underestimated complexity of taking it to a process.
Most pilots don’t reach production exactly because a loop of calls was assembled, not a managed process: it can be neither predicted in cost, nor debugged, nor driven to an SLA.
Practical takeaway for business
Orchestration is what you buy, not the model. Ask: how the next step is chosen; is state fixed; are different models used on different steps; where the process hands off to a human; how cost is visible by step. Concrete answers mean a managed process; their absence means unmanaged spend.
The effect is counted by removed coordination. Task routing, data reconciliation, status tracking — a measurable load, and exactly its reduction, not “AI adoption”, justifies the budget.
Apply this to your processes — .
Open questions
A centralized orchestrator or event-driven choreography — the choice depends on the process and fault-tolerance requirements. How finely to split steps for per-step model routing is a trade-off between savings and complexity. Where the limit of automatic coordination without a human lies is set by the error cost in the specific process.
If in your process people mostly coordinate — route, reconcile, track statuses — that is a direct candidate for orchestration. — we’ll work out steps, routing and where the handoff boundaries to humans run.