A 200 is not a contract
One of my services takes a JSON article and turns it into a published HTML page. It has a validator, because of course it does. The validator confirms the caller is authenticated, confirms the required top-level strings are present and are strings, and confirms that three arrays — sections, price rows, questions — exist and are not empty. If any of that fails it returns a 400 with a message naming the field. It is the sort of input checking you would describe approvingly in a review.
It returned 200. It returned saved: true. It published a page on which every
one of the six body sections read, in full, the word undefined.
The gap
The renderer built each section like this: take the section object, print its heading, print
its content. The caller had sent each section with its text under
body. Both sides were self-consistent and neither was wrong in isolation. The
array existed. The array was not empty. The objects inside it were objects. Every assertion the
validator made was true, and the one fact that mattered — that the renderer would read a
key nobody had put anything in — was not among them.
JavaScript then did what it is designed to do. Reading a missing property is not an error,
it is undefined. Interpolating undefined into a template string is
not an error either, it is the six-character word. So the value travelled from a typo, through
a validator that approved it, through a template that rendered it, into a database, out to a
CDN, and onto a public URL, without one layer raising an objection. The response body said
saved. It was telling the truth. It had saved exactly what it was given.
Validate the fields you consume, not the shape you receive
The lesson is narrower and more useful than validate your inputs
, which everybody
already believes they do. It is this: a validator earns its keep only where it asserts the same
facts the consumer relies on. Mine asserted facts about containers. The renderer relied on
facts about leaves. The two were never connected, so the validator was not a safety device at
all — it was a decorative one.
The practical rule I now apply is to derive the check from the read. If a template
interpolates s.content, then s.content is a required string and
belongs in the validator by that name, next to the others. Any field the renderer touches is
part of the contract whether or not anybody wrote it down, and the schema is the honest place
to write it down. Where I have a choice, I prefer a validator generated from, or at least
sitting adjacent to, the thing that consumes the data, because two hand-maintained lists of
field names will drift and the drift is silent.
There is a companion habit for the template side: refuse to render a blank. A template that
interpolates a value it was not given should throw, or substitute nothing, or leave the element
out — anything except print the name of the absence. In a language where reading a missing
key is legal, the template is the last place that can still notice, and a default of empty
string rather than undefined would have turned a published lie into a visibly
empty section somebody would have fixed in a minute.
Worse than no validation
I want to be precise about the failure, because it is not that the validation was insufficient. It is that the validation was misleading.
Had there been no validator, the first test call would have failed loudly somewhere — a missing title, an undefined array, a template error — and the shape would have been corrected in the first minute of the first attempt. Instead the endpoint gave a clean, confident, structured success. That success is what stopped anybody looking. A permissive check that returns 200 does not merely fail to catch the bug; it actively manufactures the confidence that prevents the bug from being caught. It converts a loud failure into a quiet one, and quiet failures are the expensive kind, because they are discovered by whoever finds the page rather than by whoever shipped it.
This is the same shape as a test suite that asserts a function was called rather than asserting what it produced. It goes green, and green is worse than red if it is measuring the wrong thing.
Check the artifact, not the receipt
The second habit that came out of this is about verification rather than validation, and it generalises past APIs.
A 200 is a statement about a request. It is not a statement about a page. When the output of
an operation is an artifact that somebody will look at — a rendered page, a generated
file, an email, a mobile build — the only verification that counts is fetching the
artifact and asserting something about its contents. So the deploy check after that endpoint
runs is no longer did it return 200
. It fetches the published URL with a browser user
agent and a cache-buster, counts the occurrences of the string
>undefined<, and requires zero. It also counts the rendered section cards
and requires the number I sent. Both assertions are trivial. Both would have caught this
instantly.
That check is worth more than the validator I would have written in its place, because it tests the thing the reader gets rather than the thing the server thinks it did. And it does not care how the bug arrives. A key rename, a template refactor, a schema migration, a caching layer serving a stale build — all of them show up in a fetched artifact, and none of them reliably show up in a status code.
What I would tell someone building the same thing
Three things, in order of how much they buy you.
Assert on the rendered output, always, and put that assertion in the same script that does
the publishing so it cannot be skipped. Make the required-field list in your validator match
the field list your renderer reads, and prefer a mechanism that keeps them matched over a
convention that asks you to remember. And treat a template that can print
undefined as a defect in the template, not a defect in its caller.
None of that is sophisticated. The reason I am writing it down is that I had a validator, a schema in a prompt, a structured success response, and a public URL, and the combination of the first three is precisely what stopped me reading the fourth.
Written by Liana Grigory, also written Liana Grigoryan — entrepreneur, software engineer and U.S. Army veteran in Los Angeles. More at Writing.