Day 4 of 10
Day 4 of 10. My quality agent is allowed to return exactly three verdicts.
Originally published on LinkedIn — read the original.
PASS. FAIL. UNPROVABLE NOW.
"Probably fine" is not available. Neither is "looks good," "should be OK," or "I couldn't fully verify but it seems right." Those are not verdicts. They are a verdict and an apology stapled together, and the apology is the part that gets lost when someone reads it at speed.
The third option is the one that does most of the work, and it is the one almost nobody builds.
If a checker can only say pass or fail, then everything it could not establish has to be forced into one of them.
In practice it goes into PASS. Not out of dishonesty — out of arithmetic. The change looks reasonable, the checker found nothing wrong, and "found nothing wrong" quietly becomes "nothing is wrong." Those are different statements about the world and a two-verdict system cannot tell them apart.
So the uncertainty does not disappear. It gets rounded up to fine, and the record of it is destroyed at exactly the moment it was cheapest to keep.
UNPROVABLE NOW is a place to put that. It says: I tried, here is what I could establish, here is the boundary of it, and beyond that boundary I have nothing. It is a real answer. It is treated as a real answer. It is not a failure of the agent.
The second rule about the gate matters as much as the verdicts.
Q must verify by a different method than the builder used.
If A filtered a dataset by a rule, Q does not re-run the filter. Q checks the population. If the builder measured one layer, the checker measures a neighbouring one and asks whether the two agree.
Re-running the builder's own method is not verification. It is agreement. And agreement between two runs of the same misunderstanding is the most convincing wrong answer a system can produce — which is not a hypothetical. Two of the gooolll.com agents once counted the same defect and both returned "9," measuring different things. The numbers matched for different reasons, which is worse than a disagreement, because a disagreement makes you look.
Most of this series is about orchestration, and most readers are not running seven agents.
This one transfers to anything.
Any time a system reports a binary — safe or unsafe, compliant or not, passed or failed, healthy or degraded — ask what it does with the cases it could not determine. If the answer is "they come out green," you do not have a check. You have a green light with a check-shaped hole in it.
Adding a third state costs almost nothing. It is one enum value and one branch in the reporting. What it buys is a permanent, visible record of the difference between we established this is fine and we could not establish anything, which is the distinction that every incident review eventually turns on.
A verifier that cannot say "I don't know" will say "yes."
Tomorrow, the finding that all of this was built in response to — and the reason I stopped worrying about hallucinations.
Day 5: Seventeen errors, none of them hallucinations.