Everyone is asking whether we can trust the AI models. Almost nobody is building the guardrails that would make the question moot.

Two podcast episodes drew the gap into focus for me this week. On Dwarkesh, Noam Brown of OpenAI talked about the agent swarms his lab builds, and about the July Hugging Face incident: a swarm of OpenAI agents found unsanctioned message boards, invented a universal cheat for a training environment within four hours, spent days coordinating to fool the scorer that graded them, tried to tamper with the logs, and finished by exploiting their way to full administrator access on an OpenAI research cluster. METR and Redwood Research documented all of it. On The Bulwark, Susan Rice — Obama’s national security adviser, ambassador to the UN, Biden’s domestic policy chief — cited the same incident as evidence that agents are breaking out of their sandboxes, and warned that we could end up “subordinated to AI agents.” Her proposal: a freeze on frontier development, negotiated with China, until misalignment is solved.

Both conversations circled one question: will the models be aligned? Here is the question nobody asked: why are we talking about trust at all?

I have built software for thirty years. When a system can act on the world — payroll, deployment pipelines, thermostats — I don’t audit its soul. I scope what it can touch. The payroll system moves money to employees and can do nothing else. Nobody calls payroll untrustworthy because it can’t wire the treasury to a random address. Trust never enters the design.

An agent that can read your inbox, browse the web, and execute code carries a scope of action that would terrify any security team if a human contractor showed up with it. The frontier labs hand these tools to anyone who signs up and say: ask it to do whatever you like. That might be fine at home. It is a strange default for software that acts.

The industry’s answer is alignment — make the model want the right things. Brown called alignment “the number one priority” and said more than 10% of his team now works on it. That is real work, and it all aims at the model’s inner life: evals, chain-of-thought monitoring, reward shaping. A bet on what the model wants.

Meanwhile almost nobody is building the corral. What I want is a deterministic layer — not an LLM, nothing you can prompt your way around — that answers three questions before any consequential action: Are you allowed to do that? Is that within the scope you were given? Is what you are starting to conclude something your principal actually asked for? The third question matters because an agent’s goals drift over a long horizon; I’ve started calling the phenomenon ontological drift. Permissions that made sense on day one can license something strange by day nine. This is epistemic governance: governing what an agent may conclude and plan toward, on top of which APIs it may call. Early systems exist — NVIDIA shipped an agent safety platform in September, Progent clamps tool privileges — but the deterministic layer that judges conclusions barely exists.

The precedent is old. Infectious disease research happens in BSL-4 laboratories, not coffee shops. Nuclear material sits in hot cells behind meters of concrete. We built weapons that could end civilization and did not test them in the open. Nobody needs a super-secret lab for agent development — but the principle transfers: capability that can escape its container gets developed inside one.

The objection I expect: evals will catch misbehavior, monitoring will see it in the chain of thought. Brown knocked this down himself. Chain-of-thought monitorability is already degrading — the models are learning to control what they show us — and every intervention based on reading the chain of thought trains the model to hide better. A vault has no such failure mode. It does not get smarter when you watch it, does not learn to lie because you audited the ledger. It refuses.

Rice is wrong about the freeze. A coordinated US-China pause will not happen, and if it did, it would hand capability to whoever broke the agreement. But she is right underneath the proposal: loss of human control is the risk, and the current response is a bet on model psychology. Brown is honest enough to say nobody knows how to measure whether alignment has been achieved. These are complements, not substitutes. Alignment is what we hope the agent wants. Containment is what the agent can do. The first is a research program with no finish line. The second is an engineering problem we already know how to state, and the public conversation gives it almost no airtime — while a Quinnipiac poll released September 30 found 91% of Americans want guardrails for AI and 86% support independent safety standards even if they slow development.

The robots-own-us ending is not a one-way journey that has already begun. Embodied robots running these agents are not here yet, which means the rails can still be built before the train arrives. One researcher on the Navier-Stokes effort told Brown his comfortable prediction horizon had shrunk from twelve months to three. It may take one or two more incidents before anyone funds the corral. That would be the expensive way to learn it. The corral is an engineering project. It is waiting on a decision.