Stateless services

A service is stateless when any instance can handle any request. Nothing it needs lives only in one instance's memory or local disk. Stateless services are easy to scale out. You add instances, register them with the load balancer, and you are done. If one dies, the others carry on.

State still has to live somewhere. The work is moving it into components built to hold it.

The big idea

Every piece of state has a better home than a process's memory. A process can be restarted, redeployed or replaced at any moment, and its memory disappears with it. Shared components keep state across those events. The table below shows where each kind goes.

Walking through it

Here is the classic bug. A server stores login sessions in a local map:

// Sketch: an Express-style app. Breaks as soon as there are two instances.
const sessions = new Map<string, Session>()

app.post("/login", (req, res) => {
  const id = crypto.randomUUID()
  sessions.set(id, { userId: req.body.userId })
  res.cookie("sid", id).send("ok")
})

app.get("/me", (req, res) => {
  const s = sessions.get(req.cookies.sid) // undefined on the other instance
  if (!s) return res.status(401).send("log in again")
  res.json({ userId: s.userId })
})

Run two instances behind a load balancer and trace one user:

POST /login  → instance A stores session abc
GET  /me     → instance B: no session abc → 401 "log in again"

Sticky sessions at the load balancer hide the bug by sending each user to the same instance. That creates a new problem. If instance A dies, everyone pinned to it is logged out, and the load balancer cannot rebalance freely. Moving sessions into Redis, or into a signed cookie, fixes the root cause.

Common mistakes

• Storing uploads on local disk. A second instance cannot see the file.
• Using sticky sessions as the fix. They hide the state and make failover messy.
• Keeping correctness-critical state in memory. A local cache is fine if a miss only costs time.

Check yourself

1. Why do sticky sessions only half-solve the session problem?

   Answer: losing the pinned instance logs out its users, and the load balancer cannot rebalance freely.

2. Which kind of state can safely stay in a process?

   Answer: a local cache, as long as a cache miss only makes things slower and never wrong.

Key takeaways

• Stateless services scale out by adding instances and can lose any instance safely.
• Move sessions to a shared store or signed tokens, files to object storage, and background work to queues.
• Sticky sessions hide the problem and add failover risk.
• Local caches are fine when a miss is only slower.

Further learning

Next, see the order in which a growing web app adds these pieces, in the canonical scaling path.