Every serious platform I have worked on, in payments or in e-commerce, had a version of itself that was smaller than you would guess from the outside. Somewhere in its history sits a single machine doing everything at once. This is the first half of the ladder that gets it from there to something you could actually trust: reliable, and fast, in one place. What happens after that, when one place stops being enough, is a different problem entirely, and it gets its own piece.
- Six decisions, made roughly in this order, take a platform from one fragile machine to something reliable and fast, before the word "scale" enters the conversation.
- Each one solves the failure the previous stage created, and each one adds a new failure mode the next stage has to answer for.
- Getting this half right is what earns you the right to worry about the harder half next.
01One Server, Doing Everything
At the beginning, there is one machine. It answers web requests, it answers API requests, it holds the database, and if there is a cache at all, it lives in the same process. A user types a domain name; a directory service, DNS, which is really no more than a phone book for machines, hands back an address; the browser or the app connects straight to that one box and gets a page or a JSON response back. It is a small, honest, understandable system, and for a genuinely small amount of traffic, it is also the correct one. Nobody should apologise for starting here.
The limits show up quietly at first. One slow query blocks a page load for every other visitor, because everything is fighting for the same CPU and the same disk. And there is exactly one machine standing between your users and nothing working at all. That second fact is the one that eventually forces the next decision.
02The Database Moves Out on Its Own
The first real architectural decision most platforms make is embarrassingly simple: stop making the web server and the database share a machine. Put the database on its own box. Now each tier can be resourced, tuned, and eventually scaled on its own terms, instead of one greedy query starving both jobs at once.
This is also usually the first moment someone asks "relational or not?" The honest answer is that a relational database is still the right default for most businesses, because it has forty years of tooling, transactions, and people who know how to operate it behind it. A non-relational store earns its place when you genuinely need sub-millisecond access at huge scale, when your data has no natural relational shape, or when you are storing so much of it that a single relational instance becomes the wrong tool for the job. Reach for it because the workload demands it, not because it sounds more modern.
03Vertical Scaling Hits Its Ceiling
For a while, the answer to "we need more capacity" is simple: buy a bigger machine. More CPU, more memory, a faster disk. This is vertical scaling, and its whole appeal is that nothing about your architecture has to change. That appeal runs out for two unrelated reasons. First, hardware has a ceiling; the biggest machine cloud providers will rent you is finite, and eye-wateringly expensive well before you reach it. Second, and more important, one big machine is one point of failure wearing a more expensive suit. If it goes down, everything goes down with it, no matter how much RAM it had.
The alternative is horizontal scaling: instead of one bigger machine, more machines doing the same job. It costs you something real, coordination complexity you did not have before, but it is the only path that scales past a certain point, and the only one that gives you redundancy for free. Every decision from here on is horizontal scaling applied to a different layer of the system.
04A Load Balancer Removes the Single Front Door
Once there is more than one web server, something has to decide which server answers which request, and hide the individual servers from the outside world entirely. That is a load balancer's job. Users connect to one public address; the load balancer spreads their requests across a pool of servers behind it, health-checking each one on a short interval and quietly routing around any that stop answering, often before a human notices anything is wrong.
This is also the moment public and private addressing starts to matter as a real design decision, not just plumbing. The individual web servers no longer need a public address at all, only the load balancer does. That is a smaller, cleaner attack surface, and it is the first hint of a pattern that shows up again and again in resilient systems: put a boundary in front of the thing you actually care about, and let the boundary absorb the churn.
05Replication Copies the Data, for Two Different Reasons
A single database is still a single point of failure, and by now it is also a bottleneck, because most applications read far more than they write. Replication solves both problems with the same mechanism: one primary database accepts writes, and one or more replicas hold a copy of the data and serve reads. Reads spread out across the replicas; writes still go through the primary, because somewhere in the system there has to be one place that decides what the truth is.
Replication is not instant. A replica lags behind the primary by some small amount of time, which means a read immediately after a write can, in the wrong architecture, show stale data – a user updates their address and the next screen shows the old one, because it happened to be served from a replica that had not caught up yet. And if the primary itself goes down, promoting a replica to take its place is not a light switch. The replica may be slightly behind, the promotion has to be coordinated, and the gap between "the primary died" and "a new primary is fully caught up and serving writes" is a real window of risk that every serious system has to design for, not paper over.
06A Cache Stops Asking the Database Twice
Even with replicas spreading the read load, most of those reads are asking the database the same question over and over: the same product page, the same account summary, the same piece of pricing that has not changed in the last hour. A cache sits in front of the database and answers the questions it has already seen, in memory, far faster than any disk-backed store ever will.
A cache is not free correctness, it is a deliberate trade of freshness for speed, and every cache needs an expiry policy that someone actually thought about. Set it too short, and you are back to hammering the database, just with extra steps. Set it too long, and users start seeing prices or availability that are quietly wrong. There is a sharper version of the short-expiry problem worth knowing by name: a popular key expires, and a thousand requests arrive in the same instant, all miss the cache together, and all hammer the database at once trying to repopulate the same answer. It is called a cache stampede, and the fix, letting only one request rebuild the value while the rest wait or serve the stale copy a moment longer, is a small piece of coordination logic that a surprising number of production outages turn out to be missing. And because a cache lives in memory, one cache node going down loses everything it was holding, which is why a real deployment runs more than one, in more than one place.
Nothing above makes a platform bigger. It makes it trustworthy in one place: no single machine can take the whole thing down, and no request waits longer than it has to. That is the floor everything else stands on.
Reliable and Fast Are Not the Same Problem as Big
I have built versions of this first half more than once: a payments platform in Pakistan where a small oversight could touch thousands of accounts, a search engine in Japan built for a hundred-millisecond budget. In neither case did "we need to handle more users in more places" come first. What came first was almost always some version of this exact ladder – stop trusting one machine, stop asking the database the same question twice, stop letting one slow dependency ruin every request. Scale, when it eventually arrived, was a different and genuinely harder problem, built on top of a foundation that had to be solid before it was worth having.
Every decision above solved exactly one failure and created exactly one new one. Redundancy at every tier costs coordination; speed costs freshness. There is no version of this ladder without trade-offs, only better and worse ones.
None of this required a large team or a large budget to start. It required noticing which failure was about to happen next, before it did.
A platform that is reliable and fast in one place is not yet a platform that can serve the whole world. Those turn out to be genuinely different problems, and conflating them is where the next five stages of trouble usually start.