Build with AI and trust what you make. Free guide, no signup

Everything you hand a model becomes material

A bad example is not read as a thing to avoid. It is read as text that is available.

Everything in a document you hand a model is available to it, including the parts you wrote as warnings.

A phrase turned up in the rejected list here that had never been anywhere near the work. It came out of the quality document, from a list headed what a vague decision looks like. A phrase written as a diagnosis gets read as a value, lands in the rejected list and picks up a log entry saying it was sharpened.

Worked examples do the same thing. A small model will return well formed and completely untrue objects by refilling the policy's own examples as if they were answers. The first run only looks correct if the example happens to match the move it made.

So write worked examples in a different domain from the work, with an instruction never to copy them. A policy that demonstrates its log format using the work's own field names and values gives a weak model no way to tell the format from the answer. Rewritten in an unrelated domain, both failures stop.

This class of failure is silent. Valid structure, correct shape, confident false content. An enforced schema guarantees shape and never truth.

Know which documents reach which call

A system prompt written for one step will sabotage another that reuses it. A refine policy opens by saying the object must never contain the document's content and closes by demanding the complete object and nothing else. Prepend that to the call that writes the document and every one of those calls is a contradiction, because writing the document asks for exactly the thing the policy forbids.

Sometimes the instruction wins, sometimes the standing policy does. One run here returned an object where the finished document should have been.

The fix is a separate system prompt per step. Enforce it structurally rather than by being careful: know which documents reach which call and write the absences down deliberately. The absences are part of the design rather than an oversight in it.

A guard belongs in the code or in the prompt, never both

List every prohibition in a prompt when they are all already enforced in the code and the model reads the list as evidence that the safe answer is no. Here it returned nothing on three cases out of three, including the prompt's own worked example.

Strip the prohibitions and both real cases work again. It then breaks the other end, producing a true, general, irrelevant answer on a control case, because the intended question and the wrong reading of it are both honest readings of the same English.

No third wording fixes that. A fair amount of time can go into looking for one. The resolution is arithmetic again: do not ask the question at all unless the pass added something, which is countable.