More thinking made the model worse.

That is the result I’m verifying right now from my level-two benchmark suite, and I want to write it down before I explain it away. The level-one benchmark ships tomorrow and its results are clean. Level two is where it gets strange.

Some context on what these benchmarks measure. They’re the measurement half of semantic and epistemic governance, the ideas behind the last few posts here. Semantic governance asks whether a model can hold a complex system of interaction rules — identities, permissions, who may act on what, what happens when two rules conflict. Epistemic governance asks something different: hand a model twenty or thirty sources of information, some duplicating each other, some unreliable, some solid, and ask whether it can tell them apart. Can it weigh a claim by who said it? Can it discount what it shouldn’t trust?

Those are the questions to answer before handing an agent real authority. A permission system tells you what an agent can do. Nothing tells you what it will believe. The benchmarks exist to measure exactly that gap, and to measure it without touching the real world: every problem is abstract on purpose, so nothing can be answered from memory or a guess.

Level one is a set of abstract logic problems, same design principle as my logic suite. Each problem hands the model a written rule system and a task, then grades whether the output follows the rules. A single problem is solvable by a careful human with an hour and a pencil. The full set is not solvable by a human at all — you can’t hold thirty interlocking rules in your head while checking each one. That’s deliberate. The difficulty is mechanical, so the scores measure rule-holding and nothing else.

The frontier tier clears it. The Opus-class models, Sol, Astra — solved cleanly, no problems. The budget tiers don’t, which is the story the last post told: the compliance cliff, the gap between the tiers, widest exactly where the stakes are highest. Level one says the frontier models are ready for this layer. Level two asks what happens when the problem stops being a rule system and starts being a pile of partially trustworthy information.

So level two is much harder. Twenty to thirty sources per problem instead of a handful. Sources that duplicate each other but disagree on one detail. Sources that are plainly unreliable. Sources that look reliable and aren’t. The suite doesn’t just ask the model to follow rules — it asks the model to work out what’s true from a pile of semi-trustworthy inputs, which is the thing you’d have to do yourself before trusting an agent’s report on anything real.

That’s where the odd result shows up. Run a frontier model at high reasoning effort and it scores better than the same model at extra high. More thinking budget, worse score. I’ve re-run it enough times to know the pattern is real, and I’m still verifying it — re-checking the scoring scripts, re-deriving problems, comparing transcripts line by line, the same triple check I promised before publishing any numbers. But the direction has held so far, and it’s the opposite of what the effort dial exists to do.

My working theory is that this is the complexity limit. There are two limits on these models. The first is the one where a model stops being able to solve a problem at all; the frontier tier still clears level two more often than not, so we’re not there. The second is the one where extra reasoning stops buying accuracy and starts costing it. Somewhere past a certain size of source list and rule system, the additional thinking isn’t holding the problem together better. The model re-derives a conclusion it already had right, or over-weights one subset of sources, or spends its budget on a branch that leads nowhere — and the extra effort buries the failure instead of surfacing it. The answer at extra high reads exactly as confident as the answer at high. Only the score is different.

If the pattern survives verification, it matters for anyone running agents. The effort dial is treated as a safety knob: turn it up when the stakes are high. This result says the dial has a peak, and past the peak you pay more for worse. Level-two problems — twenty sources, conflicts, duplicates — look a lot like what real deployments feed an agent every day. On those problems, the maximum-effort setting may be the wrong setting.

The level-one numbers publish tomorrow with the release. Level two stays under verification until the triple check is done. I went looking for the complexity limit, and I think I found it: the place where extra thinking starts to hurt.