The Infinite Last Mile

Prior was almost ready for a very long time. Not broken or unfinished, just almost ready. This is a special phase in software development where every major feature works, the app looks real enough to show people, and the remaining task list somehow remains constant as the files grow.

I would fix the last bug, run another review, and discover three more improvements. I would make those improvements, clean up the code around them, and uncover an architectural concern that had apparently been waiting behind the sofa. The closer I got to shipping, the more the finish line behaved like a mirage.

This was particularly embarrassing because Prior is an app about calibration. It asks you to assign a probability to a statement, then measures whether your confidence matches reality over time. When you say 80%, are you actually right about 80% of the time? I had built an app to teach intellectual humility and become completely miscalibrated about my own codebase.

The Review Hydra

Prior went through three major code reviews before I was willing to call it shippable. Each review found legitimate improvements, which is exactly the frustrating part: the feedback was not wrong.

There was no moment where I could dismiss the report, declare myself a misunderstood genius, and heroically press the release button. Each review improved the app and lengthened the remaining list, because a code review is a hydra with a GitHub account. Close one issue and two architectural suggestions grow in its place.

At some point, engineering rigor becomes another mask for procrastination.This is the trap: if every review makes the software better, then stopping feels irresponsible. There is always another pass available and another hypothetical failure that can be prevented if I sacrifice one more weekend to the gods of static analysis. Eventually, I realized that “better” and “ready” are not the same word.

A Calibration Problem About Calibration

The scoring model inside Prior forced me to confront that uncertainty is not binary, a truth I had conveniently ignored while building it.

A potential bug is not simply “possible,” and therefore blocking. It has a probability, an impact, a likelihood of detection, and a cost to repair. A one-in-a-thousand chance of silently corrupting a leaderboard is different from a one-in-a-thousand chance of a transition animation stuttering for half a second. If I keep polishing forever, I never have to choose which risks are acceptable. I can tell myself I am being thorough when I am really avoiding the moment where the software leaves my laptop and starts disagreeing with me in public.

One hundred percent confidence is rarely justified by limited evidence. Prior literally withholds its main calibration metric until a user has completed enough predictions to make the number meaningful. It understands that certainty without evidence is noise.

So its a bit funny that I wanted perfect evidence that the app was safe to release before releasing it to the only environment capable of producing that evidence (circular reasoning with a loading spinner?).

What actually matters

None of this means quality is optional. “Just ship it” is excellent advice until the thing being shipped calculates money, stores private data, or tells a user that they are statistically worse than their friends.

Prior has several boundaries where a defect would break trust, and I feel my time on those were well spent. The scoring math has to be correct. The app separates per-prediction feedback from long-term calibration, and it waits for enough samples before displaying aggregate results. It uses precision arithmetic because a tiny rounding error that changes a leaderboard positions is terrible.

The data path has to be reliable. Predictions are stored locally so the app works offline, then synchronized when the network returns. Account changes, retries, migrations, and interrupted syncs all create opportunities for a user’s history to disappear or attach itself to the wrong state.

Those failures are silent, difficult to reverse, and aimed directly at the reason someone would trust the app. They deserve paranoia.

I eventually decided on a rough heuristic to over-engineer the boundaries where the user gives you trust, and remain merely competent everywhere else.

If the score can be wrong, block the launch. If a prediction can vanish, block the launch. If a schema migration can strand existing data, block the launch. If the button is two pixels emotionally misaligned, consider going outside.

The Things That Could Be Slightly Wrong

Most software decisions are reversible.

Copy can change. Layout can change. A clumsy abstraction can be refactored after it has survived contact with an actual use case. Even an onboarding flow can be replaced without migrating every byte of user history. This does not make those things unimportant, merely non-blocking.

The Zero-User Benchmark

For most of Prior’s development, the user base was me and a few testers.

This created a wonderful testing environment because everyone involved already knew how the app was supposed to work. To put it another way, we were the least useful possible approximation of a normal user. At some point, continuing to polish stops reducing risk and starts protecting me from information. Shipping is how the next part of the learning process begins. Users reveal where the mental model breaks and which edge cases exist only in the anxious imagination of the developer. I am (unfortunately) not capable of using my own software for the first time.

Ready Is a Decision

I no longer define “ready” as the absence of things I could improve. Ready means the core trust boundaries are verified. The remaining risks are understood rather than ignored. Most importantly, ready means the value of more private polishing has fallen below the value of public evidence.

The most reliable way to achieve zero production bugs is to achieve zero production.

Perfect software has zero users. I would rather build useful software.