Limit responses and quotas

The limiter's algorithm matters less than what the client sees. Well-behaved clients do what you tell them. A clear response lets them slow down before they hit the wall and back off instead of hammering your API.

The big idea

When a client is over its limit, return HTTP 429 Too Many Requests with a Retry-After header. Many APIs also send the client's current budget on every response, so clients can slow down early. A quota is a different kind of limit. It caps total use over a long period, such as a month, and reflects a commercial agreement.

Walking through it

A refused request looks like this:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 12
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1767225612

{"error": {"type": "rate_limit_exceeded", "message": "100 requests per minute. Retry in 12 s."}}

The X-RateLimit-* names are a convention, not a standard. GitHub uses them, with Reset as a Unix timestamp. Other APIs send seconds instead. The IETF HTTPAPI working group is standardizing RateLimit header fields to remove that ambiguity. Check the current draft before you design a new public API, then document what you pick and stay consistent.

Some response rules:

• Use 429 when this client is over its own limit. Use 503 Service Unavailable when the server is overloaded for everyone. Clients and retry libraries treat them differently.
• Make the error body machine-readable, so an SDK can tell a rate limit from an exhausted quota.
• Clients should honor Retry-After, and otherwise back off exponentially with jitter.

Quotas versus rate limits

A rate limiter that loses its counts at a restart causes a brief burst. A quota that loses its counts changes what a customer is billed. That is why quotas are aggregated from usage events and stored durably.

Common mistakes

• Returning 429 for server overload. Use 503 when the server is struggling for all clients.
• Undocumented or ambiguous headers. Say whether Reset is a timestamp or seconds.
• Treating quota exhaustion like a rate limit. A Retry-After of 12 seconds is wrong when a monthly quota is used up.

Check yourself

1. When should a server return 429, and when 503?

   Answer: 429 when this client has exceeded its own limit; 503 when the server is overloaded for every client.

2. Why are quotas stored more durably than rate counters?

   Answer: a quota is a billing record over days or months, so losing it changes what customers pay. A rate counter that resets at a restart costs little.

Key takeaways

• A refused client should get a 429 with Retry-After and a machine-readable error body.
• Budget headers let well-behaved clients slow down before they are refused.
• Use 503 for overload, and 429 for a single client over its limit.
• Quotas are long-period, durable, commercial limits, and they are built differently from rate limits.

Further learning

The IETF draft linked in the rate limiting overview is the current reference for standard header names. Next, see why LLM APIs limit tokens as well as requests in token limits for LLM APIs.