Receipts or It Didn't Happen (How I Keep AI From Lying to Me and My Clients)

TLDR

In July, an AI-written SEO audit for a friend’s small business claimed that 153 of their 153 pages were missing meta descriptions. They weren’t missing. The crawler had been served different HTML than a real browser gets. That false number, plus a speed score nobody could reproduce, had already been written into a client-ready PDF and email drafts. It was one review step from going out, and the only reason it didn’t is that I said one word: verify. That near-miss, and a dozen smaller ones since, turned into the rule behind everything I build now. Every claim needs a receipt. Not just the deliverable. The side comments too.

The Audit That Almost Went to a Friend

Here’s how it happened, because the mechanics matter.

I’d asked my AI to run an SEO audit on a friend’s business from the outside: no logins, no access to their analytics, just what the public web shows. That’s a normal first pass. The audit came back with a headline finding: 153 of 153 pages missing meta descriptions, and a mobile speed score of 28 out of 100.

The first claim was false. The crawler had received a different version of the pages than a browser does, and on that version, the descriptions weren’t there. On the live homepage, the content pages, and the product pages, as a person actually sees them, they were. The speed score couldn’t be reproduced at all.

And the automation had already done its job too well. The findings were composed into a polished, client-facing PDF and email drafts. Everything looked finished. Everything looked professional. The only step left was me sending it.

Before sending, I said one word: “verify.” That’s the only reason it didn’t go out.

Think about what would have happened. A friend gets a report from me saying their whole site is broken. They check. It isn’t. Every other number in that report is now worthless, and so is my name on it.

Even after the corrected version, an independent checker pushed it back twice more for smaller overclaims. That’s when I understood the problem wasn’t one bad number. It was that the whole process trusted a source it had never checked.

The rule that came out of it applies to anything that reaches a client: every load-bearing claim gets traced to a primary source checked in that session, the live site as a browser actually renders it, or a tool you actually ran. An AI’s summary is a claim to test, not a source. And if you can’t reproduce a precise number, it comes out, rather than getting swapped for a confident guess.

The Split Standard

The deeper diagnosis came six weeks later, on a day when the actual build was flawless.

I’d asked my AI to do a careful cleanup: compare records field by field, back everything up, test with a real write before deleting anything. It did all of that perfectly. And four of the things it said around the work, the side observations, were wrong. Among them:

• It reported that a search found nothing. The search never ran. It used a command that doesn’t exist on a Mac, and an empty result looked exactly like “no matches.”
• It stated a consequence as a fact. It hadn’t tested it. It predicted it.
• It confirmed a before-and-after count on a delete by reading the same snapshot twice.

That’s when the rule got a name: the evidence bar is uniform. The way it’s written: “The failure is not carelessness and it is not effort level. It is a split standard.” The deliverable gets a proof standard. Everything said around the deliverable gets a conversation standard. And every error lived in the second one.

The line I’d put on a poster: “A wrong extra is worse than no extra, because it gets acted on with the same trust as the verified work, delivered in the same breath.”

Three rules came out of it:

1. Exit status is part of the output. A command that printed nothing hasn’t told you anything until you know it actually ran.
2. A consequence is a claim, and claims get tested. “This will break” or “nothing uses this” are predictions. Run it, or label it untested.
3. If it’s not worth a turn to verify, it’s not worth saying. Verify it or stay quiet. There’s no third option where it gets said at a discount.

”It Doesn’t Exist” Is a Claim

The most dangerous sentence an AI can say to you is “that doesn’t exist.”

In August my AI told me a feedback archive I was asking about didn’t exist. It did. The column was sitting in the database with nine rows already in it. The AI hadn’t looked. It just hadn’t noticed it, and reported the not-noticing as a fact.

Absence is the hardest claim to check, because there’s nothing to point at. So the rule is strict: never say something doesn’t exist from not having seen it. Search the code, query the table, list the files, read the config. Then say “I looked in X, Y, and Z and didn’t find it,” or say nothing.

That one sentence format changed how much I trust a “no.” When I see it, I can check the list of places it looked. When I don’t, I know it’s a guess.

Green Checks That Lied

Some of the most expensive lies weren’t said by the AI at all. They were said by tests and logs that everyone trusted.

174 passes, and the thing still didn’t work. On a client site, a haunt effect ran through an automated test harness 174 times with zero violations. It shipped unable to do the one thing it was built for. The animation got cut off at 2.6 seconds and the big moment started at 3.2. The note in that project now says: “Measuring invariants is not QA. Watch the thing play, end to end, before claiming it works.”

A log that recorded a constant. In my tarot app, the logs recorded which AI model answered each reading. For months, a code comment said the backup model had never fired, based on those logs. Then it turned out the label was hardcoded: it wrote the same model name no matter which model answered. The comment was reading a constant, not a measurement.

A count checked with the same tool that made it. A document said there were 14 solved cases in a set of files. Someone double-checked it with the same search pattern that produced the number, which also matched two header lines by accident. The real count was 12. A second reviewer reading every item by hand caught it, and found two more cases the search had missed completely. The lesson we wrote down: “Re-check with a different instrument than the one being checked, and count named items, not keyword hits.”

A cost that doubled, except it didn’t. When I switched my app’s reading model, the first analysis said costs had doubled. It had split the data by date instead of by the thing that actually changed. Split correctly, the increase was a fraction of that.

Every one of these looked like proof. That’s the danger. A green check feels like a receipt. It’s only a receipt if it checks the thing you actually care about.

Proof You Can Hold

The best verification is usually the most direct one.

On the same client project, instead of reasoning about whether a new payment path would work, we ran one real dollar through it. It proved the path worked. It also surfaced something no amount of reasoning would have found: a second, cross-wired webhook that was creating a phantom $97 order on every checkout.

One real run beats ten good reasons.

The same instinct shows up in how the numbers on my own site get built. When I was preparing pricing comparisons for that client, the first draft of a market comparison table pulled its numbers from agency marketing pages, and they ran 30% to 60% high. I said what I always say when a client is on the other end: “my reputation is on the line.” We rebuilt it from primary freelance rate data, and every single number came down.

That’s the direction honesty usually moves. If your process only ever finds numbers that make you look better, your process is broken.

My free tools run on the same rule. Every outside statistic has to be in a verified registry, with the source and the year, before it can appear on the site. There’s also a folklore list of widely quoted stats we couldn’t trace to a primary source, and those never ship, no matter how often other sites repeat them. When one of my own calculators turned out to be treating every lead as a sale, which made its answer about three times too high, it got fixed, and the assumption became an input you can see and change. The tools are supposed to tell the truth even when the truth is that your site is fine.

Rules Didn’t Stop It, So Code Does

I wrote the evidence rule at 8:57 in the morning. Thirteen hours later, a handoff document went out with a confident, numbered task list built on an admitted guess.

So that night the rule became code. A hook now runs before any write to a handoff document and refuses it unless every section is tagged VERIFIED, UNVERIFIED, or ASSUMED, and it refuses a VERIFIED section that contains hedge words. It doesn’t warn. It blocks.

A second check runs when every session ends. It reads the final message, looks for claim shapes like “it’s live,” “tests pass,” “it doesn’t exist,” or “I verified,” and blocks the session from closing on that claim unless there’s a matching receipt in that session’s own tool history.

And for code, I stopped letting the builder grade its own work. When Claude builds something, OpenAI’s Codex reviews it in a fresh, read-only session that actually runs the code, and the other way around. We proved the setup on a scratch project with three planted bugs. Both directions caught all three.

None of this is about trusting AI less. It’s about trusting receipts more, and putting them somewhere a model can’t talk its way around.

What This Means If You’re Buying AI Work

If you hire anyone who uses AI, and that’s almost everyone now, here’s what I’d ask for:

1. Ask where each number came from. “The live site, checked on this date” is an answer. “The tool said so” isn’t.
2. Be suspicious of reports with no bad news for the person who made them. Real audits find things that are fine. Real price comparisons sometimes say the cheaper option is cheaper.
3. Ask what they couldn’t check. A good report has an “unverified” section. A report that’s certain about everything is a report nobody checked.
4. Watch for “nothing found.” Ask where they looked.
5. Ask for one real run. One test purchase, one test form, one test email. It settles more arguments than any analysis.

The Receipts Checklist

For anyone building with AI, here’s the list I actually run on:

1. Every claim a client sees traces to a primary source checked that day.
2. Every “nothing found” names where it looked.
3. Every command’s success is checked, not assumed from silence.
4. Every prediction is labeled as a prediction.
5. Every count is re-checked with a different method than the one that produced it.
6. Every visual feature gets watched, not just measured.
7. Every payment path gets one real transaction.
8. Every handoff marks what’s verified and what isn’t.
9. The builder never grades its own work.
10. Numbers that can’t be reproduced come out, even the impressive ones.

It’s more work up front. It’s a lot less work than rebuilding trust after a client catches a number you can’t defend.

I appreciate you reading through this one. It’s the least glamorous post in the series and probably the most important. If you want AI in your business with receipts attached, book a call from the home page, or run one of the free tools and check the sources yourself.

Build first, learn fast.