The Orchestrator

An orchestrator is never working with one AI tool. The working environment is an ecosystem of products, open-source projects, runtimes, APIs, agent frameworks, coding environments, memory systems, skills, plugins, workflow tools, and perfectly ordinary software infrastructure. All of it is already available, and none of it was designed as a set. At the time of this writing, no vendor or open-source project ships anything you could install and call a complete orchestration system. There is no product that arrives with the inference engines, the retrieval, the memory, the tools, the identity, the policy layer, the durable execution, and the governance record designed to work together, just as a database arrives with its storage engine, query planner, and access control designed together.

So the job is to understand what each piece does well, what it cannot do, how it stores state, what it can reach, and how it fits with everything else. Chapters 3 and 4 built a taxonomy: the components of a delegated intelligence, the dispositions, the permission envelope, and the shared formats that are starting to standardize pieces of them. This chapter connects that taxonomy to the actual market. Each survey section that follows takes one component layer and asks the same questions of every product in it: what does it supply, what does it leave to somebody else, how does it store state, what can it reach, and who can stop it.

That does not require memorizing every product, which would be impossible anyway. It requires being able to look at a system and ask useful questions. Is this an editor with an agent inside it, a terminal coding agent, a persistent runtime, a task coordinator, a worker, a workflow engine, or a control surface? Does it work interactively or in the background? Does it preserve state? Can it create a branch, open a pull request, run tests, or deploy? Can it call other agents? What does a human have to approve? And — the question that matters more every month — does this product sit in one category or span several? A tool like LangGraph is simultaneously a workflow runtime, a durable-state engine, and an approval-gate mechanism; if your mental model assigns it only one of those jobs, you will use it wrong. The categories themselves are still being invented, which means the questions have to interrogate the categories, not just the products.

This survey is not an analyst’s market overview — no quadrant, no vendor ranking, no maturity model. Building an exhaustive survey of every available component in 2026 would require an encyclopedia-sized set of volumes and a staff of twenty people updating it every two days to stay current. That is how fast this market is moving: companies and open-source communities are still defining not just products but whole new categories every week. And it is not a buyer’s guide. The goal is not to recommend a stack. It is to give an orchestrator enough technical literacy to recognize real differences in approach and capability, and to keep asking useful questions about both tools and categories as they evolve, before deciding what to connect, what to delegate, and what to keep under direct human control.⁶

Step back and look at what is happening in this market right now. This period has a name in the economics of technology: an irruption. The bicycle industry went through one in the 1890s:

The 1890s bicycle boom. Between 1890 and 1896 American bicycle production went from roughly forty thousand machines to more than a million, built by more than three hundred manufacturers, and by one count nearly a third of all patent applications filed at the U.S. Patent Office in that decade were bicycle-related — competing frame geometries, tire standards, gearing systems, and brake designs, most of them incompatible with one another. Everyone, everywhere, was inventing — and then the boom peaked in 1896 and was effectively over by 1900, collapsing first amid overproduction, market saturation, and price competition. The consolidation, when it came, was not a standards body. The automobile later absorbed much of what the bicycle movement had built — the road infrastructure, the manufacturing talent, the attention — and the period of invention ended because a new platform consumed it.⁷ Carlota Perez calls the first phase of each technological revolution irruption, the years in which “intense funding of innovation … yields not only new inventions but entire industries,” before the market consolidates around the survivors.⁸ Generative AI reached that point in the middle of this decade. This chapter is a survey written during that period, and like every survey written during an irruption, it should be read as a snapshot of a moment, not a map of settled territory.

1.2.1 A ladder of coordination, not a maturity model

A caution before the survey. Multi-agent is not automatically better than single-agent, and more coordination is not automatically more mature. A single agent can handle very complex work, and that single-agent approach can itself be something an orchestrator designs. What the list below describes is a range: how much coordination different levels of agentic work require. If reading this book convinces you to overcomplicate your problem, stop — the lowest complexity that reliably does the job is the right amount. The vendor documentation converges on the same advice: compose a small set of patterns and use the lowest complexity that reliably meets the requirement.⁹ The useful skill is recognizing how much coordination a given piece of work actually needs, because the components in the survey that follows are scattered across the whole range.

The ladder itself, in ascending order of coordination cost. The first rungs describe how most people first touched these systems; the later rungs are where orchestration begins:

• Direct model calls and chat interfaces — the first step, and the one almost everyone started with: one bounded transformation, one conversation at a time. Guardrail: schema validation and a quality check on the output.
• Single agent with tools — one control loop owns the task and the tool choice. Guardrails: least privilege, step and cost limits, approvals, tracing.
• Agentic development environments — the second step, arriving roughly 2024–2025: editors, terminals, and coding agents that chain tool calls, hold a session, and do bounded work in a repository. Guardrails: review gates, sandboxing, workspace isolation.
• Deterministic sequence — the order is known and each step has a defined contract. Use code, not model choice, to decide the order. Guardrails: typed state, idempotency, retries, explicit failure paths.
• Router or handoff — a specialist takes over based on intent or domain. Guardrails: a handoff contract, a context filter, escalation and fallback.
• Manager with agents as tools — one agent owns the final result while specialists do bounded work. Guardrails: a bounded subtask schema, source attribution, aggregation checks.
• Parallel fan-out and aggregate — independent work reduces latency or increases coverage. Guardrails: isolation, concurrency and budget caps, deterministic merge.
• Evaluator-optimizer — quality criteria can be tested and revision is valuable. Guardrails: an external rubric, a maximum number of iterations, an acceptance threshold, human escalation.
• Durable event-driven execution — work spans minutes to months, waits on humans or systems, and must survive failure. Guardrails: idempotency, durable timers, correlation IDs, compensation and rollback.

Two readings of the ladder are useful. Read as a timeline, it approximates how the industry actually climbed: direct model calls and chat interfaces first, agentic development tools second, and orchestration at the rungs that enable parallel fan-out, aggregation, and durable event-driven execution — the rungs where the system, not the person, starts coordinating the work. Read as a snapshot of a single production system, every rung shows up: a retrieval call may live one rung away from a planner that lives three rungs away, and a developer session may sit on a lower rung than the production workflow it informs. That is why the survey that follows refuses to treat “orchestration tools” as one category. The components an orchestrator connects do not all live at the same rung.

One question from the ladder belongs in every survey section that follows, because it decides whether a system needs another moving part at all: does this step actually need nondeterministic judgment, or could a tool, a prompt inside an existing agent, or a deterministic transform do the same job? The full form of that question — what unique capability, context boundary, or authority boundary a proposed agent owns, and why that responsibility cannot be done by something simpler — belongs to Chapter 7. Here it is the survey’s organizing discipline: every component below has to earn its place by doing something the other layers cannot, and the survey sections are ordered so that an orchestrator can see what kind of coordination each product is actually built for.