Moving tenants and sharding

When one database can't hold every tenant, you shard. In B2B SaaS the natural shard key is the tenant ID, so each tenant's data sits on one shard. Nearly every query then touches one shard, and a tenant's transactions stay local. Simple hashing breaks down for big tenants, so you also need a directory that maps tenants to shards, and a safe way to move a tenant from one shard to another.

The big idea

Create many more logical shards than machines. Map each tenant to a logical shard with a hash or a lookup table. When you need to rebalance, move whole logical shards between machines.

Tenant sizes follow a power law, so one enterprise customer can be bigger than a thousand small ones together. Hashing alone eventually creates a hot shard. A directory (a lookup table from tenant ID to shard) lets you place big tenants on purpose, and move one without rehashing anyone else.

Walking through it

Notion is a well-documented example. In 2021, it sharded its Postgres monolith by workspace ID into 480 logical shards, spread across 32 physical databases with 15 logical shards each. It chose 480 because that number divides evenly in many ways, which makes rebalancing simple. In 2023, it described expanding to 96 physical databases by moving logical shards, not individual rows.

480 logical shards / 32 databases = 15 per database
480 logical shards / 96 databases =  5 per database
Expanding means moving whole logical shards, not rewriting rows.

A directory lookup then looks like this:

tenant "acme"    -> directory says shard 7   (a big tenant placed on purpose)
tenant "tiny-co" -> hash -> logical shard 213 -> physical database 4

Moving a large tenant online follows a standard sequence:

1. Copy a snapshot of the tenant's rows to the target shard.
2. Catch up using change data capture from the source, replaying writes that happened during the copy.
3. Verify row counts and checksums.
4. Cut over. Briefly block writes for that tenant, for seconds. Apply the last changes, flip the directory entry, and unblock.
5. Clean up the source after a safety period.

The same sequence moves a tenant from the pool to a dedicated database when it upgrades, or to another region for data residency. Building this tool early turns many future problems into routine operations.

Common mistakes

• Hashing straight to physical servers. Adding a server then reshuffles almost every tenant. Use logical shards.
• Ignoring the power law. One huge tenant on a shared shard becomes the hot spot. Place big tenants with the directory.
• Cutting over without verification. Compare counts and checksums before you flip the directory.
• Building the move tool late. The first emergency move is the worst time to write it.

Check yourself

1. Why use 480 logical shards on 32 databases rather than 32 shards?

   Whole logical shards can move between machines later. 480 divides evenly, so you can regroup into 32, 96 or other counts without splitting shards.

2. Why does a directory beat pure hashing for a huge tenant?

   The directory places the big tenant deliberately, and it can move that tenant without rehashing every other tenant.

Key takeaways

• Shard by tenant ID so each tenant's data lives on one shard.
• Use many logical shards, and map them onto fewer physical databases.
• A directory handles big tenants that hashing places badly.
• Build an online move tool with copy, catch-up, verify, cut over and clean up.

Further learning

Notion's engineering post describes how it sharded Postgres by workspace, and what it learned about rebalancing.

Herding elephants: lessons learned from sharding Postgres at NotionWith an effort to make Notion faster and more reliable for years to come — we migrated Notion’s PostgreSQL monolith into a horizontally-partitioned database fleet.Notion

Next: cost per tenant, which shows how to tell which tenants are profitable.