Rate limiting · Guide

A rate limiter caps how much work a client can make your system do in a period of time. It sits on almost every public API. GitHub gives authenticated users 5,000 REST requests per hour. Stripe rejects bursts with HTTP 429. LLM APIs meter both requests and tokens per minute.

Each number hides four questions: which algorithm counts, where the count is enforced, how a shared counter stays correct across many servers, and what the client is told when refused.

Why rate limit at all

Each goal leads to a different limit.

Abuse protection. Five login attempts per minute per account makes guessing a 6-digit code take months, not seconds.
Fairness. One tenant's runaway loop should not starve everyone else.
Cost control. A search over a billion documents or a GPU call is expensive. A limit stops one free user from running up a large bill.
Protecting dependencies. Limit your outbound calls too, such as to a partner API with its own quota.

Keep one distinction clear. A rate limit is a policy about a client. Load shedding reacts to the server's health, and it is covered in resilience.

The algorithms in one line each

Every algorithm answers one question: may this request proceed now? They differ in how they treat bursts, which are many requests in a short time even when the average is fine. Fixed windows are cheap but allow double bursts at a boundary. Sliding windows smooth that out. Token buckets allow a burst up to a capacity, then a steady rate. Leaky buckets queue requests and drain them at a fixed rate.

Where to enforce, and on what key

Limits can live in the client SDK, at the edge or API gateway (cheapest, since the request never reaches your app), in the service for business rules, or outbound toward partners. Most systems use more than one.

The key is the identity you count against. A bad key makes the limit useless or harmful.

Key	Good for	Pitfall
IP address	Unauthenticated endpoints, floods	Many users share one carrier NAT; IPv6 gives attackers many addresses, so key on the /64 prefix
User or account	Fairness, pricing	Needs a login first, so it can't protect the login endpoint
API key	Developer platforms	One customer can hold many keys, so also limit per account
Endpoint	Expensive operations	Usually combined with another key

Most real limits are composite, such as account:42:POST /exports.

Concepts in this topic

Read them in order. Each one builds on the last.

Step	Concept	What it answers
1	Window counting limiters	How do you count requests per period, and where does it break?
2	Token bucket	How do you allow a steady rate with bursts?
3	Distributed limiter on Redis	How do 200 servers share one count without races?
4	Local plus global limits	How do you avoid a Redis call on every request?
5	Responses and quotas	What does the client see, and how is a quota different?
6	Token limits for LLM APIs	Why is a request a poor unit of cost for LLMs?

Common mistakes

Keying on IP alone. Shared IPs punish innocent users, and IPv6 lets attackers rotate addresses.
Reading, computing, then writing on Redis. Two servers can read the same count and both allow. Use one atomic script.
Leaving the failure mode unplanned. Decide in advance what happens when the limiter store is down.
Counting requests when costs vary. A 150,000-token prompt costs far more than a 200-token one.

Key takeaways

Rate limits serve abuse protection, fairness, cost control and dependency protection. Load shedding is a separate tool.
Pick an algorithm for its burst behavior, and say which bursts you allow.
Choose keys that match the threat, and enforce at more than one layer.
Distributed counters must be atomic and use one clock.
Tell clients what happened with 429, Retry-After and budget headers.