Take the countable part off the model
When a question put to a model produces garbage, the fix is to find the part that can be counted and take that part away from the model.
Machinery built around a model ends up asking it to judge things. Is this record honest. Did it get more specific. Which claims have nothing behind them. Questions like that produce confident nonsense. The fix is always the same move.
Find the part of the question that can be counted and take that part away from the model.
| the vague question | what it became |
|---|---|
| did it remove a protected value | a substring comparison |
| did it keep the log properly | not asked at all, the tool writes the log |
| did it sharpen this decision | did only a tag change |
| which claims are unsourced | how many parts are uncovered |
| is this record honest | computed from before and after |
| did it get more specific | count the numbers, units and named parts |
| does this link point at real things | pick two from a numbered list |
| is this rejection earned | did a value actually leave |
| is this read back worth anything | did it name every part there is |
| which part did it mean | match it back to the counted list |
| could this change have had a justification | did the pass add anything |
Eleven of them here, no exceptions.
Why this is not about model size
The negative result is what makes the point. A 36B model fails these questions in exactly the same way an 8B one does. It is not a model that is too small. It is a question phrased so that the wrong answer is its natural reading.
So run the failing question on a much larger model before doing any prompt work. If it fails the same way, stop rewording and go looking for the countable part.
A claim to be finished is an opinion
A loop needs to know when it is done. Ask the model and you get an opinion. Compare the decision fields between two passes and you get arithmetic. The two can point in opposite directions.
Here the arithmetic caught a twelve pass run writing run 10: no change, the seed is finished into one of the decision fields, using it as a second log. The seed is the object. So the field it wrote that into is part of what the comparison looks at, which means every pass claiming to change nothing was changing something. The model said finished. The arithmetic said still moving. The arithmetic was right.
Nothing adversarial is going on in a case like that. A true sentence goes into the wrong container and only counting notices.
What counting is worth and what it is not
Ten checks of that kind, added over four phases, each one a fact rather than a judgement. Does every protected value still appear somewhere. Did a value actually leave. Did the pass add anything. A pass that fails one is thrown away rather than adopted, with the reason handed back so the retry is not a repeat.
| baseline | phase 2 | phase 3 | phase 4 | |
|---|---|---|---|---|
| phantom entries in the rejected list | 2 | 1 | 0 | 0 |
| false log entries | 3 | 1 | 0 | 0 |
| decisions improved | 0 | 0 | 0 | 0 to 1 |
| ending | settled, wrongly | settled, wrongly | settled, wrongly | blocked, correctly |
Read the third row before the first two. The checks took the invented entries to zero and left the quality of the decisions exactly where it was. The loop stopped lying long before it started improving anything and it never really started.
That is the honest shape of what counting buys. It makes the reports true. It does not make the work good. A clean set of reports is very easy to read as though it had.
Two things that keep this honest
The machinery should be generic and every question put to the model should be as narrow as possible. Those pull in opposite directions and both are true. Anything that gets more generic in the question gets worse, because a document that reaches a prompt gets copied rather than read.
This is not an argument for moving everything into code. What comes off the model here is the part of a check that can be counted. That is a different question from whether the work itself should be done by a model at run time, where the trade is capability against cost and the answer can easily go the other way.