Vertical versus horizontal scaling

There are two ways to add capacity. Vertical scaling (scaling up) gives one machine more cores, memory or faster disks. Horizontal scaling (scaling out) runs more machines that each do the same job. Vertical scaling needs no change to your application. Horizontal scaling needs the application to be built for it.

The big idea

Scaling up is the simplest move, because you change a machine and not your code. It keeps every operation local, so there is no network between parts of the work and no class of distributed bugs to chase. Scaling out is what lets you grow past any single machine and survive losing one. It is also where most of the design work sits.

The usual order is up first, then out. The reason is cost in effort, not in money. A bigger box is an afternoon's work, while distributing a system is months of work.

Walking through it

Scaling up goes further than many people expect. Cloud providers rent single machines with hundreds of vCPUs and multiple terabytes of memory. Local NVMe drives deliver millions of IOPS. A well-tuned Postgres or MySQL server on that class of hardware can handle tens of thousands of transactions per second, with a multi-terabyte working set in memory.

Stop scaling up when:

• You are near the ceiling. If you are on the second-largest size and growing 50% a year, you have roughly a year left.
• Cost becomes steep. The largest instances often cost more per core than mid-sized ones.
• You need redundancy anyway. One machine is one failure domain, so you need at least a standby.
• Recovery time grows. A very large database takes a long time to restore or to rebuild a replica.

Common mistakes

• Scaling up forever. You end with a single point of failure and a steep bill.
• Scaling out a stateful app without moving its state. Each new copy then holds different data.
• Ignoring recovery time. Huge single machines take a long time to restore.

Check yourself

1. Why is one large machine risky even when it can carry the load?

   Answer: it is a single failure domain, so you still need a standby. A standby is the first step toward a horizontal design.

2. What must be true before you add a second app server?

   Answer: the app must be stateless, or the second server will not see the first server's sessions and files.

Key takeaways

• Vertical scaling is simple and keeps work local, but it has a ceiling and one failure domain.
• Horizontal scaling survives node loss and grows further, but it needs statelessness and partitioned data.
• The usual order is scale up first, then add replicas and caches, then shard.

Further learning

Horizontal scaling depends on keeping state out of the process. The stateless services page shows where that state goes, with a bug to look for.