The moment I understood what we had built was not during a benchmark or eval. It was during a commit that took three tries.

I asked an agent to make a change. It finished, staged the work, and tried to commit. The pre-commit hook ran a scan and blocked it on a reliability finding. The agent read the finding, understood it, and fixed it. It tried to commit again, and this time its own fix had introduced a second blocking finding. It read that one, fixed it too, and ran again. This time it passed, and committed the whole change set.

I didn’t intervene. I sat there and watched it run into the pre-commit hook until it was allowed to pass.

That loop is unremarkable. Linters and static analyzers have been doing this for years. With GenAI-based findings it stops being unremarkable, because the loop only works if the gate says the same thing every time you ask – and generative analysis, on its own, does not.

A ruler that changes length

Think about measuring a board with a tape measure that is a slightly different length on every pull. You cut, you check, you cut again, and you never converge, because the error you are correcting for and the error in your instrument are the same size and you can’t tell them apart.

That is what a non-deterministic check does to an agent. An agent working against a check does something people rarely do: it asks the same question dozens of times in a few minutes and steers on the difference between the answers. It changes one thing, re-runs, and reads what moved. Changes another. And during this time, the gate that needs to be cleared has moved. You might get lucky and have no gating findings, or you might get unlucky and have multiple gates at once.

If the answers move on their own, there is no difference left to read. The finding that blocked you at 10:04 is gone at 10:05 without you having fixed it, and a new one has appeared somewhere you never touched. The agent can’t tell whether its change worked, because the ground moved at the same time. It thrashes, or it decides the problem is solved when it is not, or it routes around the check entirely, which is what people do too when they can’t trust their tools.

Determinism is what makes the loop converge. The agent in that commit caught its own regression because it could trust that the second finding was new, caused by its own edit, and wasn’t just noise that happened to surface on that particular run.

What deterministic means

A compiler tells you the same thing about the same code every time you ask it. That property is boring right up until something automated is doing the asking, forty times an hour.

The deterministic pass reads code through the real language toolchain, the same machinery an IDE uses to answer “what does this identifier actually refer to.” Where it can resolve a call site, it resolves it, identically every time.

That buys two things. The obvious one is a stable answer. The one I care about more is that the scanner now knows the difference between “this is fine” and “I could not tell.”

Saying “I could not tell”

Coverage comes back as two numbers, resolved and abstained, with a breakdown of which lanes landed where. If a check doesn’t have a way to say “I don’t know”, it will say “fine” instead. That is not a flaw in any particular scanner. It is what happens to any check that has no vocabulary for its own uncertainty.

A retriever that can’t resolve something, because dependencies are declared but not installed, or because it cannot discover a module, abstains. It doesn’t report zero findings and let you read that as a clean bill of health.

It does not replace the agentic scan

I want to be clear, because this would be easy to oversell. The agentic scan finds things the deterministic pass cannot. Judgement about intent, context that is not in the call graph, the finding that depends on knowing what this service is for. That’s real work and it’s where the harder findings still come from. The agentic scan remains the best and default scanner.

What the deterministic pass does is settle everything mechanically resolvable first, fast and cheaply, so the agent adjudicates a much smaller surface. In practice that is about 40% less scan time and about 65% fewer tokens on the agentic portion.

And you can run the deterministic pass alone, which matters in a few places: the commit or push loops, and the CI pipeline. A pre-commit hook is somewhere you are waiting on the answer, you are going to ask again in ninety seconds, and you need it to be the same answer. Fast and repeatable guardrails will capture a lot of hard to remember, hard to find issues when you run them forty times before lunch. The full scan is still there when you ask for it.

Give the loop something to aim at

Most of the conversation about agentic coding is about the generation half. How good is the model, how much context it holds, how many files it can touch at once. Much less gets said about what the agent is aiming at while it works.

An agent with no bar produces whatever quality its last instruction implied. An agent with a bar that moves learns that the bar is noise and routes around it, the same way we do. An agent with a bar that holds still can hit it, verify that it hit it, and keep going.

That commit took three tries to pass, and my coding agent took three turns without me. My application is more resilient for it.