Premature optimisation wastes effort now; the optimisation nobody scheduled fails later under pressure. Both come from leaving the question of when as a feeling instead of a number.
Two comments in the code I maintain do more work than most functions. One sits beside a public directory stored as a single document, and says to split it by ZIP prefix before it approaches the one-mebibyte document limit. The other sits beside a search that reads every listing, and says to bound it by ZIP prefix at two hundred listings.
Neither optimisation has been built. That is the point. Each comment records the size at which the simple design stops being adequate, and a note of what replaces it. I call these triggers, and I have come to think that writing the trigger is usually the correct engineering deliverable, and building the optimisation is usually premature.
The familiar failure is doing it too early. Sharding a table with forty rows, adding a cache in front of a query nobody runs, building pagination for a list that fits on one screen. Each of these has a real cost that is paid immediately and for ever: more code, more states, more ways to be inconsistent, and a product that is harder to change at exactly the stage when it most needs to change. Premature optimisation of an empty table is not caution. It is waste with a respectable name.
The less discussed failure is the optimisation that was never scheduled. Everyone agreed that the simple design would need replacing “when it gets big”, and nobody wrote down what big meant. The trouble with “when it gets big” is that it never arrives as an event. Growth is gradual. There is no morning on which the document is suddenly large; there is only a morning on which a write fails because it crossed a hard limit, and the fix that should have taken an afternoon of planned work now happens under pressure, with users waiting.
Both failures come from the same gap. The decision about when was left as a feeling, so it was either acted on too soon because it felt prudent, or too late because it never felt urgent.
A trigger is a comment, or a line in a design note, with three parts:
“Split this document by ZIP prefix before it approaches 1 MiB” has all three. “This may need optimising later” has none, and is worse than no comment, because it signals that the question was considered and answered when it was not.
A threshold should be arithmetic, not intuition, and the arithmetic should be printed rather than assumed. I have shipped rate limits with the calculation wrong directly beneath a comment asserting they were right. Writing the number is not the same as computing it.
For a hard limit, start from the limit and the size of one item. Firestore caps a document at 1 MiB, which is 1,048,576 bytes. If an entry in an aggregate document averages around 400 bytes — an illustrative figure; measure your own — the document holds about 2,621 entries. Hard walls deserve generous margin, because hitting one is an outage rather than a slowdown and entries vary in size, so I would set the trigger at roughly half that, near 1,300 entries. The arithmetic takes a minute. Not doing it means discovering the capacity on the day a write is rejected.
For a soft limit, the quantity is usually money, and the limit is a free tier. Firestore's free quota is 50,000 document reads a day. A public board implemented as a collection query reads one document per row per visit. At 200 rows, that quota is exhausted by 250 visits a day. The same board held in one aggregate document costs one read per visit, and the same quota covers 50,000 visits. That comparison is the whole argument of one document, not one query, and it is also a trigger calculation: it tells you not only which design is cheaper but at what traffic the expensive one starts costing money.
Soft limits need less margin than hard ones, because crossing them degrades rather than breaks. They do need a threshold, though, because the degradation is a bill, and bills arrive after the fact.
A trigger is only useful if someone can tell when it has been reached. The tempting way to find out is a job that periodically counts things. In a metered database that is the wrong answer, because counting by reading costs one read per item counted, and a scheduled count scales with exactly the growth it was built to watch. I described that trap in the scheduled job you cannot see fail: a watchdog that scans a collection is part of the bill.
The better approaches cost nothing or nearly nothing:
Triggers are much easier to set when every feature arrives with its cost stated. I put reads and writes per user action into the commit message for anything that touches the database: this screen costs one read on open, this action costs two writes, typing in the search box costs nothing because filtering happens over what was already fetched.
Once that number exists, the trigger is usually derivable from it. Reads per action, multiplied by the expected actions, compared with the free quota, gives the traffic at which the design needs to change. Without the number, nobody can defend the design and nobody can say when it will need defending. A feature whose cost nobody wrote down is a feature nobody can reason about later.
Because the trigger carries the next design, reaching it is undramatic. Someone reads the comment, sees the one-sentence plan, and schedules the work with the headroom the margin bought. By then the product has usually changed enough that the plan needs adjusting, which is itself a good reason the optimisation was not built earlier: it would have been built for a product that no longer exists.
Occasionally the trigger fires and the right answer is to move it. Entries turned out smaller than estimated; traffic is lower than feared; the free quota changed. That is fine. Adjust the number, write down why, and leave the simple design in place. What matters is that the decision is made against a measurement.
In reviews, the question “will this scale?” tends to produce one of two unhelpful answers: a rewrite, or a shrug. There is a third answer, and it is usually the right one: not yet, and here is exactly when, and here is what we will do then.
It keeps the code as simple as the current size allows. It makes the future work visible, sized and owned. And it replaces a vague worry that recurs in every planning conversation with a number that can be checked, which is the only kind of worry an engineering team can actually act on.
Written by Liana Grigory, Entrepreneur and Software Engineer, from work on The Care Royal, Tegula Stone and Unified Savers. Everything above describes decisions actually made on those systems, including the ones that turned out to be wrong.