A prompt template is a unit of measurement

A prompt template is usually treated as a convenience. Write the wording once, fill in the parts that change, stop copying and pasting. That is true and it is the smaller half. The larger half is that a template gives every request a name, and a name is what lets you attribute anything at all.

The first three arrive with the telemetry. The fourth has to be built. All four hang off the same template name, which is what lets you read them together.

What goes into one

The instruction. What you are actually asking for, in the fewest words that are still unambiguous. It is on this list because a list of everything else would be odd, and because every item below it is something people leave out.

A name and a version. Everything below depends on this one. If the template is not identified in what gets logged, nothing downstream can be attached to it and you are back to a monthly total.

The purpose, written down. What the answer is for and what a good one looks like. This is not the instruction and it is not for the model. It is for the test that checks the output later, and for the next person to change the template.

The policy constraints. The rules the business already has about what may be said, done or disclosed. Do not invent these. Somebody whose job it was has usually written them down already, and borrowed constraints are testable in a way that invented ones are not.

A clear split between fixed and filled. The fixed wording is the thing under test. The filled part is the data. If the two are tangled you cannot tell whether a bad answer came from the wording or from what was passed in.

The output shape. Hand it a schema rather than asking for prose, so the answer can be checked by something other than a person reading it. See ask for a shape, not prose.

Worked examples, where anything is being extracted. They stop it inventing values that were never there. See examples keep extraction honest.

What it lets you measure

Four things hang off the template name, and only the first three arrive on their own.

Cost per template. Which design decision is expensive, rather than which month was expensive. A total tells you to worry. A per-template number tells you where.

Latency per template. Which one is slow. Latency is paid by the person waiting rather than by the account, so it never shows up on the bill at all.

Which model answered. The tier becomes a decision you can see rather than one that was made once and forgotten. See which model, once you have decided it is AI.

Quality per template. Whether the constraints in the template held. This one has to be built, because nothing produces it for you. See the cost is measured, the quality is not.

Write every use down

A number on a dashboard is not a measurement. It has to land somewhere it can be asked questions later, which in practice means a row per request in a database.

What the pipeline knows. The template name and version, which model answered, what the call cost and how long it took. All of that is available at the moment the call returns and none of it can be recovered afterwards.

What the response knows. Whether it succeeded. Whether it had to refuse. Whether it hit one of the constraints in the template. Whether it took a retry to get there. The response carries outcomes the infrastructure never sees, and pulling a few of those out into their own columns costs almost nothing at the time and is impossible later.

Supporting tables around it. Rates per model, so a cost can be recalculated rather than frozen at whatever the price happened to be that day. Quality results per template and version, so the quality side ends up with a history in the same shape the cost side already has.

That last one is what makes the section below possible. Cost has a time series because it always did. Quality only gets one if somebody kept the results.

It is a system of record, not a guarantee

Writing it down does not make the output good. It makes it possible to notice when the output stopped being good and to say when that started. Those are two different things and it is worth being clear about which one you have.

None of this stops a bad answer going out. It means that when somebody asks how long it has been happening, there is an answer rather than a guess.

And every pipeline is different

There is no schema here to copy. What is worth recording depends on what the pipeline actually does and on who is going to ask about it later. A pipeline that drafts text and one that makes decisions about money want different columns and different tests.

The part that carries across all of them is the template name being on every row. Everything else is local.

Read them in pairs

One number on its own is easy to misread. The information is in the pairs.

Cost down and quality down. Usually a cheaper model on a template that needed the expensive one. The saving is real and it was not free.

Latency up and quality flat. Something was added to the context that is not earning its place.

Cost up and quality up. Fine, if the answer is worth what it now costs. That is a decision to make rather than a problem to fix.

Everything flat and the total climbing. Volume rather than design. Look at how often the template fires, not at what is in it.

Why the name is the whole thing

Free text prompts give you one number at the end of the month and nothing to attach it to. The same work behind a named template gives you a row per template, and every question worth asking is a question about one of those rows.

A template without a name is just a prompt, and a prompt is not something you can measure.