← Back to blog

· Timothy O'Brien

Governing Agents as Their Understanding Changes

A maintenance agent notices that two machines keep failing at the same time. Its diagnostic procedure treats each machine independently. An investigation finds a shared controller. The system has learned something useful, but acting on that discovery now raises a different question: who has authorized work on the controller and the second machine?

That is the problem behind my new paper, The Orchestrator: A Lifecycle Architecture for Self-Developing, Self-Operating, and Self-Improving Agentic Systems. It is now available on Zenodo as an architectural preprint, version 0.2.9, under Creative Commons Attribution 4.0.

The maintenance case is a constructed example. It makes a distinction that matters whenever we delegate continuing work: discovering a wider dependency can require a better model, a new capability, and an explicit decision about scope. A successful diagnosis does not issue that decision.

The model is part of the operating contract

An ontology is an explicit description of the concepts and relationships a system uses to represent its domain. In this example, it describes machines, controllers, and which machine depends on which controller. Those definitions influence the system’s decisions. A reset that appeared to affect one asset may turn out to affect several.

Authentication establishes who is making a request. Access control evaluates whether the requested operation is permitted. The orchestration problem also includes the meaning used to formulate that request: which systems count as part of the task, what evidence supports the proposed action, and which consequences require another approval.

The paper proposes governing changes to those explicit representations. An agent can report an unfamiliar situation and propose a new interpretation. Adopting that interpretation requires review of its consequences for capabilities, evidence, state, and authority. The mechanism governs operational decision inputs and execution boundaries; it does not claim to control every thought inside a language model.

Three responsibilities, with a controlled path between them

OPERATE performs approved work, monitors outcomes, preserves evidence, and flags situations that need intervention. IMPROVE investigates deficiencies and possible limitations in the current model, considers new concepts, and proposes changes. DEVELOP builds and tests capabilities against the proposed model. These are responsibilities within a system, and can involve software services, agents, analytical models, and people.

The orchestrator coordinates the handoffs. Independent controls determine whether a candidate can become an operating release. Where a proposal exceeds delegated scope, the designated human authority must decide which additional decisions can be automated and under what conditions.

The release binds the domain model and its dependencies to the capabilities, instructions, authority rules, evaluations, and compatible operational state. Approval refers to that identified package. At the receiving system, an action must still satisfy its authorization checks. An old worker retaining yesterday’s model does not regain permission simply because it missed a deployment notification.

How the paper connects to the books

The three books approach this work from different positions. Book 1 addresses the person who recognizes a limitation and routes the problem. Book 2 addresses the engineer who builds the contracts and enforcement paths. Book 3 addresses the organization that owns the authority, funds the work, and decides when to stop. The paper gives the lifecycle connecting those responsibilities a more precise technical specification.

It builds on existing work in autonomic computing, ontology evolution, bounded autonomy, and governed capability deployment. Its contribution is the connection across a complete change cycle: a change in meaning becomes a reviewed release, and that release remains tied to the authority used when an action occurs.

This version is a preprint and has not been peer reviewed. It supplies a reference architecture, illustrative interface contracts, analytical examples, and an evaluation plan. It reports no complete operational implementation or measured effectiveness. The next research step is to build and compare the proposed controls with simpler approaches, including ordinary schemas and policy gates.

If you are building a system that improves its own procedures, start with one change it could propose. Identify which definitions would change, which capabilities depend on them, whose approval would extend the scope, and how a stale worker would be prevented from acting. Those are the obligations this paper is intended to make inspectable.

Read the preprint on Zenodo or inspect the public manuscript and supporting contracts. Later revisions will remain connected through the all-versions DOI.

The Orchestrator Series

The professional, the system, and the organization behind delegated intelligence.