Case studies
Case study: notification system
Designing push, email and SMS delivery at scale, covering the pipeline, preference checks, idempotency, provider rate limits, retries and priority lanes.
A notification system takes an event, such as "your order shipped" or "Alice commented", and delivers the right message to the right user on the right channel: push, email, SMS or in-app. It rarely needs clever data structures. It is a reliable pipeline with validation, preferences, fan-out to channels, retries and observability.
Almost every product grows one of these. Chat alerts, feed mentions, security codes and marketing campaigns often share infrastructure, with careful isolation so a Black Friday email blast can't delay login codes. Third-party providers are unreliable partners with their own rate limits. Your job is to be a good citizen upstream, and a stubborn, idempotent client downstream.
In an interview, look for the pipeline diagram, preference checks before enqueueing, per-channel queues, idempotency keys, provider rate limits and dead-letter queues. Strong answers also separate priority lanes and discuss digests against real-time sends.
Concepts in this section
| Step | Concept | What it answers |
|---|---|---|
| 1 | Notification pipeline | What are the stages from event to provider? |
| 2 | Preferences and quiet hours | Should this message be sent, on which channel, and when? |
| 3 | Idempotency for notifications | How do retries avoid sending five codes? |
| 4 | Rate limits and provider quotas | How do you stay under each provider's limit? |
| 5 | Retries and provider failures | What happens to a message that keeps failing? |
| 6 | Priority lanes and broadcasts | How do security messages avoid waiting behind marketing? |
| 7 | Templates and localization | How are messages rendered safely in every language? |
Common mistakes
- Calling providers from the API request. Provider slowdowns then become slow requests for your users. Enqueue the work and return.
- One queue for every channel. A throttled SMS provider then blocks push. Keep a queue per channel.
- Sending before checking preferences. Check preferences, quiet hours and frequency caps before any work reaches a provider.
- No idempotency. Producers, queues and providers all retry, so duplicates are a matter of time.
- No alerts on the signals that matter. Alert first on OTP latency and dead-letter queue depth, then on marketing throughput.
Key takeaways
- A notification system is a pipeline, not a single call to a provider.
- Check preferences and quiet hours before enqueuing, and use digests for low-signal noise.
- Idempotency keys and a delivery log beat hoping providers never duplicate.
- Rate-limit per user and per provider, and isolate security messages from marketing.
- Retries need backoff with jitter, capped attempts, dead-letter queues and token cleanup.
Further learning
ByteByteGo's articles regularly cover notification pipelines and fan-out patterns used in interviews.
ByteByteGo blog (Alex Xu)Explain complex systems with simple terms, from the authors of the best-selling system design book series. Join over 1,000,000 friendly readers. Click to read ByteByteGo Newsletter, a Substack publication.blog.bytebytego.com
Amazon Builders' Library essays on queues, retries and avoiding overwhelming downstreams apply directly to provider integrations.
Amazon Builders' LibraryA collection of articles that describe how Amazon develops, architects, releases, and operates technology. Use these articles to understand how Amazon builds and maintains systems that are reliable, available, and sec…aws.amazon.com
A focused notification-service system design covers the API, temporary storage, senders and at-least-once delivery.
Start with the notification pipeline.
Related
Message queues
A durable buffer between the code that creates work and the code that does it. Start here for the map of queue concepts.
Idempotency and retries
Making operations safe to retry, with idempotency keys, deduplicating consumers and jittered backoff, so at-least-once delivery never means a double charge.
Rate limiting
Why systems cap how much work a client can send, which algorithm counts, where the limit lives and what the client is told. Start here for the rate-limiting map.