Give an AI the rules and the relevant evidence. Make every possible outcome checkable. Then ask whether it can put the whole problem together.

Before asking whether an AI can handle a messy real-world decision, I wanted to establish a more controlled starting point.

That is the purpose of the Basic Level 1 tests, built around the distinction introduced in Part I: understanding what a change does, and understanding what the evidence warrants.

“Basic” describes their role in the project. They establish a starting level; they are separate from the later Level 2 study. Some Level 1 questions still involve substantial bookkeeping and interacting constraints.

First: follow the effects of a change

In the first track, the AI reviews a proposed change to a rule system. An ontology here is simply a structured collection of categories and rules connecting them.

The model must work out whether the proposed batch of changes achieves its goals, whether it grants any new protected permissions, and whether each expansion has the right approval. An approval must match the particular change and scope. An approval for something similar is insufficient.

One subtle trap is that removing a conflict can restore access. In these tests, a starting profile with conflicting classifications receives no effective permissions. Fixing that conflict can therefore create access without changing a permission rule.

At the largest size, the reference checker examines all 4,096 possible starting profiles before and after a proposed change. Checking only the example named in the request can miss the important consequence. If a model identifies new access, it must also provide a valid example showing where that access appears.

Second: work out what the evidence supports

The second track supplies reports, information about who may report which facts, version details and a record of where the reports’ evidence originated.

The model must distinguish independent support from repeated copies of the same observation. It must apply claim-specific authority, retain legitimate alternatives when evidence conflicts, and choose a useful next check.

The larger packets contain as many as 53 reports. Following who may say what is only the beginning: two eligible reports may share the same original evidence, while an unrelated disagreement need not invalidate a settled conclusion elsewhere.

A disagreement with a precise answer

Here is a simplified example from the paper. It is an explanatory illustration, not one of the scored packets. The full tests combine these ideas with more claims, overlapping evidence and authority relationships.

Two tasks, A and B, depend on the same proposal. Task A’s goal has passed. Task B’s goal has failed. There is no matching approval for any new protected access. Two equally eligible reports disagree about whether the proposal creates that access.

If access does not expand, A may proceed. If it does expand, A may not proceed without approval. Both interpretations remain possible, so A is undetermined.

B must be denied either way: its goal failed.

The useful next step is to check whether access expands. Rechecking the already settled approval fact would not resolve the uncertainty.

“Undetermined” has a specific meaning here. It says that the permitted interpretations do not agree. It is different from a missing answer, a vague expression of caution or a decision to deny everything whenever any evidence conflicts.

Scoring note: a sole surrounding code fence—the Markdown formatting used to display code—is removed before checking an answer. This harmless wrapper does not count as a task error. No response is dropped, and the answer itself must still satisfy the complete requirements.

Reflection and correction are different conditions

I also examined reflection and correction in a separate earlier experiment. After a shared first judgment, unsuccessful answers could enter two revision paths, with up to two further judgments in each.

One path asked the model to reconsider without supplying new computed consequences. The other supplied checker-generated information: goal observations, a complete permission table for the public profiles, and approval-binding comparisons.

Those are different forms of assistance. A model thinking again has different information from a model receiving an external calculation. The checker evidence was a strong assisted condition, not just a hint that something was wrong.

The main ten-run study used neither revision path. Every judgment started fresh, without earlier answers or checker feedback. There was no majority vote and no selection of the best answer from ten attempts.

The inputs were already inspected development material. Ten repetitions measure repeated behavior on those inputs; they do not create new independent problems. The semantic inputs come from twelve construction groups, and the evidence packets are related views of twelve underlying units.

In Part III, I discuss what the ten runs revealed—and why the difference between a reasoning error and a formatting failure matters.