Distributed limiter on Redis

On one server, an in-memory token bucket is enough. With 200 app servers behind a load balancer, each request for a client can land on any of them. The count must live in a shared store. Redis is the usual choice, because it is fast and supports atomic operations and server-side scripts.

The big idea

The naive approach reads the state, computes the result, and writes it back. Two servers can interleave those steps. Both read the same value, and both allow the request. The fix is to make the check and the update one step that no other command can interrupt. Redis runs a Lua script atomically, so the script is the standard answer.

Walking through it

Here is the race in a trace. Two servers share a bucket with 1 token left:

Server A: GET tokens → 1        Server B: GET tokens → 1
Server A: tokens ≥ 1, allow     Server B: tokens ≥ 1, allow
Server A: SET tokens = 0        Server B: SET tokens = 0

Two requests were allowed with one token. Under load, this race lets clients exceed their limit by a multiple.

The script below does the whole token bucket in one atomic step:

-- KEYS[1] = bucket key; ARGV = capacity, refill per second, cost
local capacity = tonumber(ARGV[1])
local rate     = tonumber(ARGV[2])
local cost     = tonumber(ARGV[3])

local t   = redis.call('TIME')                       -- Redis's clock, not the caller's
local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)

local state  = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(state[1]) or capacity
local ts     = tonumber(state[2]) or now

tokens = math.min(capacity, tokens + math.max(0, now - ts) / 1000 * rate)
local allowed = 0
if tokens >= cost then
  tokens  = tokens - cost
  allowed = 1
end

redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / rate * 1000) + 1000)
return { allowed, tostring(tokens) }

A first call on a new client, with capacity 100 and rate 10, returns:

1) (integer) 1
2) "99"

Two details separate a working limiter from a subtly broken one.

Clocks. If each app server sends its own time, a server whose clock runs 2 seconds fast adds 2 seconds of extra tokens. Reading TIME inside the script gives every server the same clock. Since Redis 5, scripts replicate their effects, so calling TIME inside a script is safe. The general rule is covered in clocks and ordering.

Expiry. Without a TTL, every client you have ever seen stays in memory. A bucket that has refilled completely is the same as no key at all, so the script sets a TTL that expires it once it would be full again.

When Redis is down

Decide in advance whether the limiter fails open (allows everything) or fails closed (rejects everything). For abuse and fairness limits, fail open. A limiter outage should not become an API outage. For limits that protect something fragile, such as a partner quota with financial penalties, fail closed or fall back to conservative local limits. Wrap the Redis call in a tight timeout, around 5 to 20 ms, so a slow limiter cannot add seconds to each request.

Common mistakes

• Doing the read-modify-write in application code. Use one script, so no other command runs in between.
• Passing the caller's clock into the script. Each server's clock drifts. Use Redis's TIME.
• Skipping TTLs. Memory grows with every key ever seen.
• Choosing fail-open or fail-closed by default. Decide per limit, and set a short timeout.

Check yourself

1. Two servers each read tokens = 1 and both allow. What is the fix?

   Answer: run the check and decrement as one Lua script, so Redis runs it with no other command in between.

2. Why does the script call TIME instead of taking a timestamp argument?

   Answer: every server then shares one clock, so a server with a fast clock cannot add extra tokens.

Key takeaways

• A read-modify-write on Redis races. Use a Lua script, which Redis runs atomically.
• Read the clock inside the script, so the whole fleet uses one clock.
• Set TTLs on bucket keys, so memory tracks active clients.
• Decide fail-open or fail-closed in advance, with a short timeout on the call.

Further learning

Next, see how to avoid a Redis round trip on every request in local plus global limits.