Measure what you ship, not what you wrote

Reading code tells you what should happen. Fetching the page tells you what does. These disagree more often than is comfortable, and almost every interesting bug lives in the gap.

Here are four checks that work on the output, what each one found, and — importantly — the false positive each one produced first.

1. Run the accessibility engine against rendered HTML

Not against components in a test renderer. Against the actual HTML your server returns.

Most accessibility rules are about relationships between elements: is this heading level a jump from the last one, is this landmark unique on the page, does this label point at something that exists. A component is correct in isolation and wrong once composed, and only the composed output can tell you.

We ran an engine across one page of each template. 93 violations. The most interesting was not a violation at all in the source: a roadmap page whose status columns were styled to look like headings but were divs containing badges. Visually a heading. To anything navigating by heading, invisible.

*The false positive:* the first run reported a serious error on four pages. Those four were pages where we had guessed a URL slug that did not exist, so we had audited the not-found page four times. Every finding was real — about a page nobody visits. Always confirm the URL you tested is the page you meant.

2. Compute contrast from the tokens, not from a screenshot

Accessibility engines need a layout engine to check colour contrast, which means they are useless in a plain test environment. But if your theme is design tokens with literal colour values, the ratios are pure arithmetic. No browser required.

This found error text at 1.84:1 in our dark theme — invisible, on every form in the product.

*The false positive:* our stylesheet defines its token block twice, once for a superseded palette and once for the current one. The first version of the checker read the first block for one theme and the last block for the other, then confidently reported eleven failures, most against tokens no browser ever applies. Resolve the cascade the way the browser resolves it — later definitions win — or you are measuring a colour scheme that does not exist.

3. Fetch every URL you claim to have

Your sitemap is a list of promises. Fetch all of them.

Watch for the soft 404: many frameworks render a not-found page while returning status 200. Those get indexed, compete with your real pages, and look fine in any check that only reads the status code. Check the body for your not-found marker too.

This found an entire documentation section where every link on the index went nowhere — the individual page route had never been created, and the links were missing their locale prefix on top of that. Two independent faults, on an index that renders ten confident cards.

4. Compare what you advertise against what you register

Anywhere your app declares capabilities in one file and implements them in another — commands, routes, feature flags, translation keys — those two lists drift, and nothing typechecks the relationship.

*The false positive, and it nearly shipped:* our scan reported twelve commands declared but never implemented, which would be a user-visible "command not found". They were implemented — registered dynamically in a loop, with a name built from a template string. Static analysis cannot see through that. We checked all twelve identifiers by hand against the definitions before reporting any of them, and every one matched.

That is the discipline the whole approach depends on. A measurement you have not sanity-checked is not better than an opinion — it is an opinion wearing a number, and it is more persuasive than it deserves to be.

Why the output and the source drift

Because the build is a program too. Framework conventions insert things. Bundlers rewrite things. A file that exists in your editor might be excluded from the type check, which is exactly how we shipped a test file with two type errors: the test ran and passed, the site built, and the typecheck that CI runs was the only thing that failed. We found it a day later, in the output.

Which is the point.