4.2.1 - The Failure Whose Only Symptom Is That the Answer Looks Wrong

Decide where each kind of failure appears and what a person is handed when it does, including the ones that never go red.

Failure surfacing

When this breaks, where does it show up and what does a person get to work from?

The call

Decide where each kind of failure appears and what is handed over when it does. Include the ones that never go red, because in an AI build those are most of them.

Sharpen failure surfacing with AI

This is a Run idea, so the AI move is Run it for real. You can sharpen the decision here, but it only proves out against a system that is actually live.

New here? Let your AI do this with you. Paste your problem into Guide me and it builds a prompt that walks you through this in ChatGPT or Claude. You do not need to know how to prompt.

Prefer to drive the AI yourself? Give ChatGPT or Claude this page and ask it to help. For skill.txt, the Claude Code plugin and other ways in, see how to use vibe2value.

Or copy this prompt into AI chat, replace the bracketed lines with your real failures and keep the instruction exactly as visible here. It helps you put your running/failure.md together by refining your three lines until someone who did not build the system could act on them.

You are checking whether this failure surfacing plan is clear enough before you move forward.

Constraint:
Every kind of failure must have a named place it appears and a named thing a person is handed when it does. A failure that surfaces nowhere is a gap. Saying so counts as an answer.

Example of the standard:
Vague: "Errors are logged and we get alerts."
Sharp: "A lost capture and a missed digest both write to the log. Bad advice writes nothing at all: the sender gets a suggestion that is wrong and I hear about it only if they say so. So every capture is acknowledged by return email, which at least makes silence visible."
The sharp version names where each kind lands and what you get to work from. It also names the one that lands nowhere instead of leaving it off the list.

Working draft:
Kinds of failure: [list them, including the ones that do not crash]
Where each one appears: [log, channel, dashboard, nowhere]
What a person is handed: [what they can actually work from]

Task:
Decide whether every kind of failure has a named place and a named handoff. If one has neither, say so plainly and mark it as a gap rather than inventing a route for it.

Check:
- Is there a kind of failure whose only symptom is that the answer looks wrong?
- Could a person who did not build this act on what they are handed?
- Have you named the failures that surface nowhere, rather than leaving them off the list?

Return:
- Verdict: clear, or needs work
- The corrected three lines
- Any failure that surfaces nowhere, named as a gap

Copy this into AI chat. Replace the bracketed parts. Keep the rest unchanged. AI will likely suggest refinements based on what you enter. Use those to sharpen your thinking, not replace it.

Deciding where a failure lands is the decision. Making it land well is the craft. That is Fail well, What kind of failure is it and the other ways to run it for real.

The same idea on a recipe card

A burnt dish announces itself. You smell it from the next room, the smoke alarm goes and nobody has to be told. An under-seasoned dish announces nothing. It looks right, it is plated the same and the only person who finds out is the one eating it, who may not say anything.

Kitchens plan for both. There is a bin for the burnt one and a tasting spoon for the other. The tasting spoon exists because some failures will never set off an alarm, so somebody has to go and check on purpose.

The recipe card is just a simple example, using everyday cooking ideas everyone understands, to make the concept clear. See the recipe card.

What it really means

Most systems fail loudly. Something throws, a status code goes red and you find out. That is the failure everyone plans for, because it is the one that makes a noise.

A build with a model in it fails differently. The call succeeds, the response is well formed, the page renders and the answer is wrong. Nothing has gone red, because from the system's point of view nothing has gone wrong. The only symptom is that the answer looks wrong. It only looks wrong to somebody who already knows what right looks like.

So the work here is not error handling. It is deciding, ahead of time, which kinds of failure can happen, where each one becomes visible and what a person is holding when it does. Some of them will surface nowhere. Writing that down is the answer, not a gap in the answer.

Make failure surfacing concrete

Compare the version that sounds responsible with the version somebody could act on.

Too vague: Errors are logged and we get alerts.
Concrete enough to act on: A lost capture and a missed digest both write to the log. Bad advice writes nothing at all: the sender gets a suggestion that is wrong and I hear about it only if they say so. So every capture is acknowledged by return email, which at least makes silence visible.

The second version tells you what you would be holding at two in the morning. It also admits the one that has no route yet, which is the part most answers leave out.

Both versions come from consideredContent, the one project carried through all 35 ideas. See the project.

It did not throw. The status was fine. Every gate on the path went green, because every gate on the path checks something other than whether the answer is true. So it arrives looking exactly like a good one, and the only person who can tell is whoever knows the subject.

Check failure surfacing

Pass: You can point at where each kind of failure appears and say what a person or an AI is handed to work from when it does.
Fail: Any failure whose only symptom is that the answer looks wrong.

A silent failure you have named beats a loud one you have not.

What you'll walk away with

This post is about the running decision: what happens the moment something goes wrong in a system that is already live. You'll come out with your own knowledge-base/system/running/failure.md written and sharpened: every kind of failure named, where each one surfaces, what a person is handed and an honest list of the ones that currently surface nowhere.

Write it down

Your failure.md is a real file in your project. The AI forgets everything between sessions. It reads this file each time to pick up what you already decided, on this idea or another part of the build. Write it down once instead of explaining it again.

knowledge-base/system/running/failure.md is just a suggested name and place for this information. Call the file and put it wherever suits your project. What matters is that it is written down in a format that is easily understood by both people and the AI.

Risk and mitigation

Risk: The failures that never go red are the ones you find out about from a user, months later, after they have been happening the whole time.
Mitigation: Name one silent failure per model call or external call then decide now whether you are giving it a route or accepting it as a known gap.

Key takeaway

Do not move on until every kind of failure has a place it appears and something a person can work from, including the ones that never go red.

How to document your failure surfacing

Write your failure surfacing up in full so your AI has the whole picture: the answer, why you believe it, what you are optimising for, where you might be wrong and the standing instruction it should follow. Keep it in a file with your project like this.

# Failure surfacing

## Answer
Kinds of failure: [list them, including the ones that do not crash]
Where each one appears: [log, channel, dashboard, nowhere]
What a person is handed: [what they can actually work from]

## Evidence
Why you believe this. Incidents you have had, things users reported before you saw them, the ones you only found by accident.

## Decision
What you are optimising for and what you are accepting. Which failures you have decided to leave silent for now.

## Risk
What breaks if this is wrong.

## Standing instruction
What the AI should do every time it touches this. For example: when you add a call that can fail, say where the failure will appear and what it hands back. If the answer is nowhere, say so.