Guard against false positives

A check is only trustworthy if you act on what it says. The fastest way to lose that is the false positive, because one category that cries wolf makes you start ignoring all of the output, the accurate findings included. So part of making output trustworthy is protecting the signal, keeping the noise down so every finding is worth reading.

Say it another way

A false positive is when a check raises the alarm about something that is actually fine. A few of those and you start ignoring the alarm altogether, like a smoke detector that goes off every time you make toast, so a real fire gets missed too.

One noisy category makes you start ignoring the whole check, accurate findings and all. Tag why each finding fired so the noise can be grouped and suppressed, and the signal is trustworthy again. Like an oven timer that beeps at every little thing, the kitchen stops listening and misses the one bake that mattered, so you keep it for what counts.

Doing this with Claude Code: Guard against false positives

When Claude is checking or reviewing, the quiet killer is the false positive. A check that cries wolf trains you to skim or ignore all of its output, so the accurate findings get thrown out with the noise. Trust is slow to build and fast to lose, so a noisy category costs far more than it looks.

Protecting that trust earns its keep in five ways: one noisy category poisons the rest, mute and fix rather than tune live, show the line with examples, tag why each finding fired and watch for the check that comes back short.

One noisy category poisons the rest. When one check cries wolf you start distrusting all of them and the real findings get ignored alongside the noise. A false positive costs more than a missed nicety, because it spends trust you cannot quickly earn back.

Mute and fix, do not tune live. When a category is noisy, switch it off to restore trust in the rest, then reframe and fix it properly before turning it back on, rather than leaving it spraying noise while you tinker.

Show the line with examples. Give it an acceptable case next to a real problem, so it learns where the boundary sits and stops flagging the harmless ones.

Tag why each finding fired. Have it return the reason it flagged each one, a detected-pattern field, so false positives become data you can group and suppress wholesale rather than a vague sense that it is noisy.

Watch for the check that comes back short. A review that returns four findings where there were forty does not look like noise, it looks like an answer, so you stop looking. Ask what would be invisible to it, a call reached through a wrapper or sitting under another name, then confirm from a different direction, because what the code can do is not the same as what a run log shows it did.

Common questions

What is the opposite failure, when a check comes back short?
A check can under-report rather than cry wolf. It finds four things where there were forty, because the ones it missed went by another name or through a wrapper it never followed.
Why does a search miss things that are really there?
A name search only ever finds names, so when the thing and its name come apart the search under-reports. A wrapper can re-export something under a different name and every caller downstream uses that name instead.
Why is under-reporting worse than a check that fails outright?
Four hits look like an answer, so you stop looking. A search that fails at least tells you that you do not know, where one that comes back short quietly ends the investigation.
How do I guard against a check that comes back short?
Ask what would be invisible to the check in the first place, then get a reading from a different direction. The code tells you what can happen and the log tells you what did.

Back to For developers. The craft around it is Working with AI.