System design · Guide

Sections: 6
Pages: 285
Reading time: ~12 h
Take first: Foundations

What you will be able to do

Turn vague requirements into numbers (QPS, storage, bandwidth, latency budgets) and use them to justify every component.
Choose and configure databases, caches, queues, streams, CDNs, object stores and search indexes for a given access pattern.
Explain replication, partitioning, consistency models, clocks, consensus (Raft) and distributed transactions, and their failure modes.
Apply resilience patterns (timeouts, retries with jitter, idempotency, circuit breakers, load shedding) to keep systems up under stress.
Design multi-region, multi-tenant and event-driven architectures and articulate their trade-offs.
Run a 45-minute system design interview end to end and hit the signals interviewers look for at SDE2 and SDE3.
Design complete systems: feeds, chat, notifications, ride-sharing, payments, video, a KV store, a crawler and an LLM chat product.

System design is choosing components and the contracts between them so a system meets its requirements under real load, real failures and a real budget. Almost every decision is a trade-off between latency, throughput, availability, consistency and cost, and the job is to make those trade-offs deliberately, with numbers, rather than by copying whatever a large company published a blog post about.

This topic moves in the order you'd learn it on the job. It starts with estimation, scaling and API design, then the data and messaging building blocks every backend uses (databases, replication, sharding, caches, object stores, search, queues, streams, load balancers, CDNs). Next comes the distributed-systems theory that explains why those building blocks behave the way they do: consistency models, failure models, clocks, consensus, distributed transactions and conflict resolution. After that, the architecture patterns teams apply at scale: rate limiting, idempotency, resilience, microservices, event-driven designs, multi-tenancy and multi-region. It ends with a repeatable interview framework and 68 case studies, from a URL shortener to an LLM chat product.

Why it matters

The jump from junior engineer to SDE2 is largely a jump in design ownership. A junior engineer implements a well-specified feature inside someone else's design. An SDE2 is expected to design the service: pick the data store, define the API, estimate the load, say what happens when a dependency fails, and defend those choices in review. System design interviews exist to test exactly that, and they're usually the round that decides the level of an offer.

The course also connects the rest of the degree. It builds on computer systems foundations (memory, storage engines, networking, HTTP), and it's the foundation for the infrastructure course's work on SLOs, capacity planning and Kubernetes, for AI engineering topics like RAG and agent infrastructure, and for inference work like serving platforms.

Who it's for

You're comfortable writing backend code and have shipped something that talks to a database, but you haven't had to make a system survive a 10× traffic spike, a region outage or a sharding migration. You may be preparing for system design interviews, or you've just been handed your first design doc. No prior distributed-systems coursework is assumed; every lesson builds from first principles and links back to prerequisites.

How to study it

Pace. 285 lessons, about 12 hours of reading. Fundamentals and building blocks first, then distributed systems (the hardest material), then patterns and case studies. Don't rush the distributed-systems unit; it's what separates someone who can draw boxes from someone who knows why the boxes fail.
Do the arithmetic. Every lesson has worked numbers. Redo them on paper with different assumptions. Estimation fluency comes only from repetition.
Build small things. Theory sticks when you've watched it break. Good projects as you go: a URL shortener with Postgres and Redis (then add a read replica and observe lag); a rate limiter in Redis with a load test; a toy Raft implementation (MIT 6.824's labs are the gold standard); a Kafka pipeline that survives killing a consumer mid-batch; a webhook sender with signing and retries.
Design out loud. For each case study, set a 45-minute timer and design it yourself before reading. Then compare. Practise with a friend using the interview framework.
Use the further learning sections. Each lesson ends with the best papers, engineering blog posts and talks on its topic. Read at least one primary source per lesson: original papers (Dynamo, Raft, The Tail at Scale) and real postmortems teach things summaries can't.

How to approach a design

Every lesson feeds this sequence. Each step constrains the next, and skipping ahead is how designs end up over-built.

Clarify requirements. Functional (what it does) and non-functional (latency, availability, durability, consistency, scale). Write the numbers down.
Estimate scale. QPS, read/write ratio, storage per year, bandwidth.
Define the API. A handful of endpoints or RPCs. This pins down what data must exist.
Model the data. Entities and access patterns. The access patterns pick the database, not the other way round.
Draw the high-level design. Clients, load balancer, stateless services, data stores, caches, queues.
Deep-dive the hard parts. The hottest path, the biggest data set, the strictest consistency requirement.
Find bottlenecks and failure modes. What breaks at 10× load? What happens when a node, a zone or a dependency disappears?

Trade-off	Pushing one way	Pushing the other way
Consistency vs availability	Reject requests during a partition	Serve possibly stale data
Latency vs consistency	Read from a nearby replica	Coordinate with the leader or a quorum
Throughput vs latency	Batch work together	Process each item immediately
Cost vs headroom	Run hot, scale on demand	Over-provision for spikes
Simplicity vs scale	One database, one service	Shards, queues, many services

Roadmap

The numbered list is a shorter path. The sections below are every lesson in the topic.

Fundamentals

The vocabulary and arithmetic everything else uses.

Back-of-the-envelope estimates: turn users into QPS, storage and bandwidth.
Scaling fundamentals: vertical vs horizontal, statelessness, and the order a system grows in.
Availability, latency and the tail: nines, percentiles, composition math and tail latency.
API design: REST, gRPC, GraphQL, pagination, versioning, errors and webhooks.

Data building blocks

Where data lives and how it's split, copied, cached and searched.

Traffic and messaging

How requests reach your services and how services talk to each other asynchronously.

Distributed systems

The theory behind the building blocks: what "consistent" means, what fails, and how machines agree.

Architecture patterns

Proven patterns for keeping large systems up, fair and evolvable.

Case studies

Start with the framework, then work through complete designs. Each one reuses the building blocks above in a different combination.

The system design interview framework
URL shortener
News feed
Chat system
Notification system
Search autocomplete
Ride-sharing
Ticket booking
Payment system
Video streaming
Distributed key-value store
Web crawler
LLM chat product
Recurring deep-dive patterns: a cheat sheet that maps hard problems to proven solutions.

Course resources

The single most important book for these pages. Kleppmann explains replication, partitioning, transactions, consistency and stream processing with unusual clarity and depth; most of Data building blocks and Distributed systems map onto its chapters.

Designing Data-Intensive Applications (Kleppmann)Designing Data-Intensive Applications (DDIA) — an O’Reilly book by Martin Kleppmann (The Wild Boar Book)dataintensive.net

A free, broad overview of system design topics with diagrams and links, useful as a checklist and for quick revision:

The System Design PrimerLearn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards. - donnemartin/system-design-primerGitHub

MIT's graduate distributed systems course, with lecture videos, paper readings (MapReduce, GFS, Raft, Spanner) and the famous Go labs where you implement Raft and a sharded key-value store yourself:

MIT 6.5840 (formerly 6.824): Distributed Systems6.5840 Home Page: Spring 2026pdos.csail.mit.edu

Lecture 1: IntroductionMIT 6.824: Distributed Systems

Martin Kleppmann's Cambridge lecture series is a shorter, gentler route through the same theory (clocks, replication, consensus, CRDTs). Start here if MIT feels steep:

Distributed Systems 1.1: IntroductionMartin Kleppmann

Google's SRE book is the reference for availability, SLOs, incident response and running systems in production:

Site Reliability Engineering (Google)Go through the complete table of contents of sre Google book, outlined are the key topics and insights covered in this essential resource for SRE professionals.sre.google

The Amazon Builders' Library has some of the best practical writing on timeouts, retries, load shedding, caching and deployments, written by the engineers who run AWS:

Amazon Builders' LibraryA collection of articles that describe how Amazon develops, architects, releases, and operates technology. Use these articles to understand how Amazon builds and maintains systems that are reliable, available, and sec…aws.amazon.com

For interview preparation, ByteByteGo's step-by-step walkthrough is a compact introduction to the structure interviewers expect; pair it with the interview framework lesson:

System Design Interview: A Step-By-Step GuideByteByteGo

Fundamentals

Napkin constants

The handful of numbers worth memorizing so you can estimate scale in your head, from seconds per day to what one server can handle.

Latency numbers every engineer should know

A reference table of how long memory, disk and network operations take, and the ratios that decide where a design can afford to wait.

Back-of-the-envelope estimates

Turn users and features into QPS, storage and bandwidth numbers in two minutes, and use them to decide what the design actually needs.

The estimation chain

The five steps that turn users and behavior into queries per second, storage, bandwidth and server count, with the multipliers you should say out loud.

Worked example for a Twitter-like app

A full estimate for a timeline-based social app, from requests per second to egress cost and server count, with each number tied to a design decision.

Read/write ratio and the one-machine check

Two checks to run after the numbers. The read:write ratio tells you where to spend effort, and the one-machine check asks whether you need a distributed system at all.

Estimation practice

Three practice problems, photo sharing, chat and analytics events, each with its assumptions, arithmetic and the design conclusion it forces.

Vertical versus horizontal scaling

Scaling up with a bigger machine versus scaling out with more machines, what each costs, and when scaling up stops being the right answer.

Stateless services

A stateless service can handle any request on any instance. Learn where the state goes instead, and the login bug that appears when a service keeps sessions in memory.

Scaling fundamentals

Vertical versus horizontal scaling, stateless services, the usual order in which a growing system adds parts, and how to find the real bottleneck. Start here for the scaling map.

Redirect status codes: 301 vs 302

A redirect's status code tells browsers and CDNs whether they may keep the answer, which decides whether your server ever sees the next click.

The canonical scaling path

The usual sequence in which a successful web app adds parts, from one server to queues, shards and regions, with the symptom that triggers each step.

The scale cube

Three independent ways to split a system, by cloning, by function and by data, and how to name the next step in a design review.

Finding the bottleneck

How Amdahl's law, Little's law and utilization tell you which resource limits a system and how much a fix will actually help. The database is usually the answer.

Growing a marketplace

A worked plan for a small marketplace that expects 10 times its traffic in a year, step by step with real numbers, and why it needs no sharding.

Nines and downtime

Availability is the share of time or requests that work. Each extra nine cuts allowed downtime tenfold, and the last few nines need automation, not heroics.

MTBF and MTTR

Availability depends on how often things break and how long each break lasts. Cutting recovery time often adds more nines than preventing failures.

Percentiles and tail latency

Latency is a spread of values, not one number. Averages hide the slow requests that users feel, so describe latency with p50, p99 and p99.9.

Availability, latency and the tail

Nines, percentiles and the math of composing services. Why p99 matters more than the average, how fan-out amplifies the tail, and how to design for both uptime and speed.

Composing availability

A request that needs every dependency multiplies their availabilities, which lowers the ceiling. Redundancy adds nines only when the replicas fail independently.

Tail latency amplification

A request that fans out to many backends waits for the slowest one. With enough fan-out, a rare slow backend becomes the normal response time.

Taming the tail

You can't stop every server from being slow. Set timeouts from measured latency, send a hedged second request when the first is late, and return partial results on time.

Latency budgets

A latency budget splits one end-to-end target across every hop, so each team knows its share. Parallel calls cost the slowest one, and sequential calls add up.

API styles

REST, gRPC and GraphQL can answer the same request. Pick one by asking who calls the API and what they need from it.

Safe methods and idempotency

Each HTTP method makes a promise about side effects. Knowing which methods are safe to repeat tells you when a retry is harmless and when it could charge a card twice.

API design

An API is a contract that outlives the code behind it. Design the styles, methods, pagination, errors and async flows so changes stay additive.

Cursor pagination

Offset pagination skips rows and gets slower and less stable as you go deeper. Cursor pagination asks for the rows after the last one seen, so every page costs the same.

API versioning

The best version is the one you never ship. Make additive changes, make clients tolerant of new fields, and version only when you must break the contract.

Structured errors

A good error response says what went wrong, whether to retry and how to fix it, in a form code can read. Use status codes, a stable error code and an ETag for concurrent edits.

Webhooks and long-running operations

Work that takes minutes should return 202 at once and finish in the background. Clients then poll an operation or receive a signed webhook when it is done.

Little's law

A one-line formula that links how many items are in a system, how fast they arrive, and how long each one stays. It sizes queues, worker pools and servers.

Data building blocks

Access patterns

An access pattern is one concrete query your system runs, with its lookup shape, rate and latency target. Write the list first, and let the choice of store follow from it.

Databases and data modeling

Choose between relational, document, wide-column, key-value, graph and analytical stores by listing your access patterns first, then model the data to fit them.

Indexing strategy

An index is a sorted copy of some columns that lets the database jump to matching rows. Column order, covering and partial indexes decide how useful it is, and every index slows every write.

Query-first modeling

Key-value and wide-column stores have no joins, so you name the partition key for every query before you store anything. DynamoDB and Cassandra both work this way.

Specialized data stores

Graph, time-series, search, vector and columnar stores each win one access pattern that general-purpose databases handle poorly. Columnar storage explains why analytics reads far less data.

An order system, modeled two ways

Model the same order system once in Postgres and once in DynamoDB, using the same access patterns. The comparison shows what you gain and what you give up with each.

Single-leader replication

One node accepts every write and streams its log to followers. The choice between synchronous and asynchronous commits decides whether a crash can lose an acknowledged write.

How changes are shipped

The four ways a database describes a change to its replicas, statement, physical, logical and change data capture, and the failure each one invites.

Replication

Keeping copies of the same data on several machines, the three topologies that databases use, and the anomalies that replication lag causes. Start here for the replication map.

Failover and split brain

The four steps of promoting a follower after the leader fails, the data that asynchronous replication can lose, and how fencing prevents two leaders from writing at once.

Replication lag anomalies

Why reads from a lagging replica can show the past, the three user-visible anomalies this causes, and the standard fix for each.

Multi-leader replication

When several nodes accept writes at once, why that needs a conflict plan, and the four ways to settle a conflict, from avoiding it to merging with domain rules.

Leaderless quorums

How Dynamo-style stores accept writes with no leader, using W and R quorums, sloppy quorums with hinted handoff, and repair with read repair and Merkle trees.

Range and hash partitioning

Range partitioning gives each shard a block of neighboring keys, which keeps scans cheap but can hot-spot. Hash partitioning spreads writes evenly but scatters neighbors.

Sharding and partitioning

Split data across machines by choosing a shard key, hashing or ranging it, routing requests, handling hot spots, resharding without downtime and generating IDs that stay unique.

Choosing a shard key

A good shard key has high cardinality, even traffic, query locality and stability. Choose it from your most common queries, and plan for the single hot value that breaks even a good key.

Secondary indexes when sharded

When data is sharded by its primary key, queries on other fields either visit every shard (local index) or use an index that is itself sharded (global index), with different costs for reads and writes.

Resharding without downtime

Two ways to add capacity to a sharded system, and a dual-write playbook for moving data while traffic keeps flowing. Each step stays reversible until the final cleanup.

Cache layers

A request can be served from a cache at many layers, from the browser to the database. The closer a cache sits to the user, the more load it saves and the harder it is to invalidate.

Cache patterns

A cache pattern decides who loads data into the cache and when writes reach the database. Cache-aside is the usual choice, because the cache is optional and the app still works without it.

Hit rate and sizing

Hit rate sets how much traffic reaches the database. Going from 90% to 99% cuts database load tenfold, so size the cache for the working set and measure hit rate as a capacity metric.

Redis specifics

Redis is the default shared cache, and it stores far more than strings. Know its data structures, its single-threaded commands, its eviction settings and how cluster mode splits keys.

Caching

Where to put a cache, how to read and write through it, how to size it from hit-rate math, and how to keep it from serving stale data or melting down.

Cache invalidation

A cache learns that its data is stale through TTLs, deletes on write, versioned keys or change-data-capture. Each gives bounded staleness, not strong consistency, so choose by how stale you can tolerate.

Cache stampede

When a hot key expires, every request misses at once and runs the same expensive query. Fix it by letting one request recompute, refreshing early, or serving slightly stale data.

Cache failure modes

Caches also fail in three other ways. One key can get too much traffic, requests can ask for keys that don't exist, and an empty cache can send every read to the database. Each has a specific fix.

Memcache at Facebook

Facebook's 2013 memcache design uses leases, regional pools, commit-log invalidation and gutter pools. It is a catalog of what goes wrong when a cache becomes load-bearing.

The object storage model

Object storage keeps whole blobs under flat keys behind an HTTP API. Learn that model first, and the rest of the storage design follows from it.

Durability and erasure coding

Object stores reach eleven nines of durability with redundancy and fast repair. Erasure coding survives more failures than three full copies, at under half the storage cost.

Object storage

Store files and media in S3-style object stores. Keep bytes out of your app servers and databases, and design uploads, metadata and costs around that split.

Presigned and multipart uploads

Let clients upload straight to object storage with a signed, time-limited URL, and split big files into parts so one dropped connection costs one part, not the whole file.

Metadata and object consistency

Keep bytes in object storage and queryable facts in a database. Write intent first, confirm after the upload, and clean up what a crash leaves behind.

Storage classes and egress cost

Object stores price storage by how often you read it. Move cold data to cheaper classes with lifecycle rules, and serve hot content through a CDN, because data leaving the cloud is often the largest bill.

File sync design

A Dropbox-style sync client stores each file as hashed chunks, uploads only the chunks the server lacks, and commits versions with a base-version check so concurrent edits never silently overwrite.

Inverted index

Map every word to the sorted list of documents that contain it, so a search becomes a few list lookups and a merge instead of a scan of every document.

Text analysis: tokens, stems and synonyms

Before a word enters the index, an analyzer splits, lowercases, filters and stems it. The same analyzer must run on queries, and changing it means reindexing everything.

Search systems

How inverted indexes, analyzers and BM25 scoring work, and how to run Elasticsearch-style search at scale with sharding, indexing pipelines and hybrid vector retrieval.

BM25 ranking

How BM25 scores a document for a query, using term frequency with saturation, a rare-word weight and length normalization, worked by hand and in code.

Distributed search: query then fetch

How a search index split into shards answers one query. Each shard scores its own documents, the coordinator merges the top hits, and then only the winning documents are fetched.

Search indexing pipeline

How a search index stays in sync with its source database using change data capture, idempotent versioned writes and zero-downtime reindexing behind an alias.

Hybrid search with reciprocal rank fusion

Combine keyword ranking with vector similarity. Reciprocal rank fusion merges the two lists using only ranks, so you never compare their raw scores.

MapReduce

A batch model with two user functions. Map emits key-value pairs, the framework shuffles them so each key lands on one reducer, and reduce combines the values. Failed tasks are simply rerun.

Spark and shuffles

Spark runs a pipeline as a graph of steps and keeps data in memory where it can. Narrow steps stay on one machine, wide steps need a shuffle, and the shuffle usually decides the cost.

Batch and stream processing

How to move and transform large data sets reliably. Batch jobs run over bounded data, streams run over events as they arrive, and the same ideas of time, state and idempotent writes apply to both.

Event time, windows and watermarks

Every event has the time it happened and the time the system saw it. Aggregating by event time gives reproducible answers. A watermark says when a window is complete, and late events are handled after that.

Exactly-once in streams

A stream processor restores its state and rewinds its input after a crash. The state counts each event once, but the output can repeat. End-to-end exactly-once needs an idempotent or transactional sink.

Lambda vs Kappa

Two ways to keep analytics fast and correct. Lambda runs a batch layer and a speed layer with duplicated logic. Kappa uses one streaming pipeline and replays the log when it needs to reprocess.

Design an ad-click aggregation pipeline

Design exercise. Count clicks per ad per minute within about a minute, pace budgets, and bill exactly. A streaming path gives freshness, and a batch reconciliation path owns the money.

Consistent hashing

Place keys and nodes on the same ring, and each key belongs to the next node clockwise. Adding a node then moves only the keys on its arc, not almost every key as with hash modulo N.

Traffic and messaging

Layer 4 vs layer 7 load balancing

What a layer 4 balancer can see compared with a layer 7 proxy, and why most production systems use both.

Load balancing and proxies

How a load balancer chooses which server gets each request, and how a badly set up balancer can cause the outage it was meant to prevent.

Balancing algorithms

Round robin, least outstanding requests, power of two choices and consistent hashing, and where each one fails.

Health checks and draining

How a balancer detects a sick backend with active and passive checks, why deep health checks can empty a whole fleet, and how to remove an instance without dropping requests.

Service discovery and service meshes

How clients learn which instances exist right now, the three discovery styles, client-side balancing, and what a service mesh costs.

Global load balancing

How DNS and anycast steer a user to the nearest healthy region, and why DNS failover can take minutes.

Speed of light and round trips

Distance sets a floor on latency that no server tuning can remove. A fresh HTTPS request pays several round trips before the first byte, which is why content has to move closer to users.

CDN routing and cache hierarchy

A CDN sends each user to a nearby point of presence, then uses tiers, an origin shield and request collapsing so that a new object reaches your origin about once, not once per PoP.

CDNs and the edge

Serve content from points of presence near users. A CDN cuts latency, absorbs attacks and cuts bandwidth costs, and its cache keys, headers and purge rules decide how well it works.

Cache control and cache keys

The cache key says which requests count as the same object, and Cache-Control headers say how long to keep it. Get the key wrong and you waste the cache or leak private data.

Purging and versioned URLs

Change content in a CDN by giving it a new URL, purging exact URLs, or purging by tag. Pick the cheapest one that matches how many pages a change touches.

CDN hit ratio and cost

CDN economics come down to egress price and hit ratio. Each point of hit ratio you gain cuts origin traffic, and a jump from 80% to 95% cuts origin load by a factor of four.

Click counting off the hot path

Why a redirect should never update a counter on the links table, and how an event queue and batch workers count clicks instead.

Edge compute and security

Code that runs in CDN points of presence can reject bad requests and answer simple questions close to users. It's a poor fit for work that needs a distant database, and it also absorbs attacks.

Polling and long polling

How a client gets updates by asking, why short polling wastes almost every request, and how long polling holds each request open until there is news.

SSE and WebSockets

Server-Sent Events push a one-way stream over plain HTTP with built-in resume, WebSockets give full-duplex frames, and WebTransport adds streams and datagrams over HTTP/3. Choose by direction and message rate.

Real-time delivery

How a server pushes updates to clients that did not ask, from polling to WebSockets, and the gateway, pub/sub and reconnect design that serves millions of open connections. Start here for the real-time map.

Gateways and connection registry

Why real-time servers split into thin stateful gateways and stateless services, how a TTL registry finds the gateway holding a user, and how many connections one machine can hold.

Fan-out with pub/sub

How one message reaches thousands of connected users through a pub/sub backbone, why subscribing by interest beats routing to each user, and the write math for large channels.

Connection lifecycle

How heartbeats find dead connections, how sequence numbers and resume make reconnects lossless, and how jittered backoff prevents a reconnection storm after a deploy.

Synchronous vs asynchronous work

The difference between waiting for a task to finish and handing it off, and why that choice shapes latency, failures and user experience.

Message queues

A durable buffer between the code that creates work and the code that does it. Start here for the map of queue concepts.

Queues vs pub/sub topics

Two ways to deliver a message. A queue gives each message to one worker. A topic gives a copy to every subscriber. Real systems use both.

Delivery semantics

The three promises a messaging system can make (at-most-once, at-least-once, exactly-once) and why the real choice is when the consumer says "done".

Acks and visibility timeouts

How a broker knows a message is still being worked on, what happens when a worker disappears, and how to set the timeout so work isn't duplicated.

Message ordering

Why messages can arrive out of order, what it costs to guarantee order, and how to design consumers that don't need it.

Retries and dead-letter queues

How to retry a failing message with growing delays, when to stop, and where to put messages that will never succeed so the rest of the queue keeps moving.

Backpressure and autoscaling

A queue only stores work, it doesn't create capacity. Learn how to tell when the backlog is growing, how to scale workers on the right signal, and what to do when you can't add more.

Choosing a queue product

How Amazon SQS, RabbitMQ, Google Cloud Pub/Sub and Redis Streams differ, and how to pick one based on the shape of your work rather than on brand names.

Durable workflows

How to run a multi-step process (charge, reserve, ship, notify) that must survive crashes, retries and undo steps, using a workflow engine instead of chained queues.

The log as a data structure

An append-only, ordered sequence that many readers can consume at their own pace. Learn how a log differs from a queue, and how retention and compaction shape it.

Kafka architecture and replication

How Kafka splits a topic into replicated partitions, what makes a write durable, and how KRaft replaced ZooKeeper. Learn the pieces before you tune them.

Event streaming and Kafka

How a durable log lets many consumers read the same events at their own pace, and how Kafka builds that log from partitions, replicas and consumer groups.

Producers, keys and batching

A producer picks a partition with a key, batches records for speed, and sets acks for durability. Learn the settings that decide ordering, throughput and duplicates.

Consumer groups, offsets and lag

Consumers in a group split a topic's partitions, commit offsets to record progress, and fall behind when lag grows. Learn how commit timing sets delivery guarantees and how rebalances work.

Exactly-once inside Kafka and its limits

Kafka transactions give exactly-once processing from topic to topic, but the guarantee stops at Kafka's edge. Learn what it covers, and how sinks finish the job.

Sizing partitions and avoiding hot keys

Partition count sets your parallelism, and it is hard to change later. Learn to size it from throughput and storage, and to handle keys that carry too much traffic.

Distributed systems

Linearizability

The strongest guarantee for one piece of replicated data. Once a write finishes, every later read sees it, and no reader can go back to an older value.

CAP and PACELC

CAP says that during a network split you must choose between answering every request and returning the latest value. PACELC adds the everyday trade-off that exists even when the network is fine.

CAP, PACELC and consistency models

What "consistent" means once data lives on several replicas, from linearizable to eventual, what each guarantee costs, and how to pick one per feature.

Causal and session guarantees

Weaker promises than linearizability that still fix the bugs users notice. Causal consistency keeps related events in order, and session guarantees protect one client's view.

Isolation vs consistency

Isolation and consistency both get called "consistent", but they answer different questions. Isolation is about overlapping transactions. Consistency is about whether replicas agree on the latest value.

Choosing a consistency model per feature

Real systems mix models. For each piece of data, ask what it costs when a reader acts on a stale value, then pick the cheapest model that avoids that cost.

Partial failure and the unknown outcome

A remote call can fail in several ways that look identical from the caller's side. Learn why silence tells you so little, and why a retry can charge a customer twice.

Network faults and gray failures

Packets are lost, delayed, duplicated and reordered, and partitions can be one-way or partial. Learn the faults to design for, including gray failures that pass health checks.

System models and what you assume

A correctness proof depends on the assumptions behind it. Learn the three parts of a system model, network, node and timing, and the model real systems should design for.

What goes wrong in distributed systems

Partial failure, unreliable networks, paused processes and timeouts, and the fencing tokens and failure detectors that let you build correct systems on top of them.

Process pauses and clock jumps

A process can freeze for seconds and keep believing it is still the leader. Learn the causes of pauses and clock jumps, and why they break leases and locks.

Timeouts and failure detectors

Since you can't tell dead from slow, you guess with timeouts. Learn how to pick them from measured latency, and how phi accrual detectors adapt to a jittery network.

Fencing tokens and why locks are hard

A lock can expire while its holder is paused. Learn how fencing tokens let the storage reject a stale holder, and why the Redlock debate matters.

Metastable failures and retry storms

Some outages keep going after their trigger is gone, because the system's own reactions keep it overloaded. Learn the three parts of a metastable failure and how to break the loop.

Clock drift and NTP

Every server's clock drifts, and NTP only corrects it within the limits of the network path. A clock is an estimate with an error bar, not an exact reading.

Monotonic vs wall-clock time

Wall-clock time tells people when something happened but can jump. Monotonic time only moves forward, so use it to measure how long things take.

Time, clocks and ordering

Why wall clocks lie across machines, how skew silently loses writes, and how Lamport clocks, vector clocks, hybrid logical clocks and TrueTime order events you can't time directly.

Last-write-wins and clock skew

Last-write-wins keeps whichever write carries the higher timestamp. When clocks disagree, that is not the write that happened last, and the loss is silent.

Lamport clocks

A Lamport clock is a counter that moves forward on every event and jumps past any message it receives. It gives an order that respects cause and effect, without any physical clock.

Vector clocks

A vector clock keeps one counter per process. Comparing two vectors tells you exactly whether one event happened before another, or whether they were concurrent.

Hybrid logical clocks and TrueTime

Hybrid logical clocks combine physical and logical time, so timestamps stay near real time and respect cause and effect. TrueTime makes clock uncertainty explicit, then waits it out.

Unique ID generation across shards

Ways to create IDs that stay unique across many shards without a central counter, compared by size, time ordering and coordination. Snowflake packs a timestamp, a worker ID and a counter into 64 bits.

Majority quorums

A decision counts only when more than half the nodes accept it. Because any two majorities overlap, two conflicting decisions can never both win. This is why consensus clusters have 3 or 5 nodes.

Consensus and Raft

How a cluster agrees on one value or one log despite crashes and partitions, how Raft works, and where etcd, ZooKeeper and consensus-backed databases fit.

Raft leader election

How a Raft cluster picks one leader. Followers wait for heartbeats, randomized timeouts start an election, and voters refuse candidates whose logs are behind.

Raft log replication

How a leader writes a command to a majority before it commits, how followers are repaired to match the leader's log, and why committed entries survive leader changes.

What consensus costs

Every committed write needs a round trip to a majority plus a disk flush. Across regions, distance dominates. A single group is one leader, so you scale by running many small groups.

The dual-write problem

Writing to two systems without a shared transaction leaves a window where one write has happened and the other has not. Learn the ways this goes wrong and how to avoid the window.

Two-phase commit

The classic protocol for atomic commits across several databases. Learn how prepare and commit work, why a crashed coordinator blocks everyone, and how distributed SQL databases reduce that risk.

Distributed transactions

Keeping data consistent when one business operation spans several databases or services, using two-phase commit, sagas, the transactional outbox and reservations, and redrawing boundaries first.

Sagas and compensating actions

A saga runs a business operation as local transactions, each with an undo step. Learn how compensations work, how to order the steps, and why sagas are atomic eventually but not isolated.

Orchestration versus choreography for sagas

A saga can be driven by a central orchestrator, or by services reacting to each other's events. Learn the trade-offs, and why durable workflow engines make orchestration practical.

The transactional outbox

Update a database and publish an event without a dual write. Learn how the outbox commits the event with the change, how a relay publishes it, and how an inbox removes duplicates.

Reservations and try-confirm-cancel

For scarce resources such as seats, rooms and stock, split each step into a hold and a confirmation. Learn how TCC uses expiring holds instead of database locks.

Concurrent writes and last-write-wins

A conflict happens when two replicas accept writes that neither has seen. Last-write-wins always picks one, but it silently drops the other write and trusts clocks that drift.

Conflict resolution and CRDTs

What to do when replicas accept concurrent writes. The options run from last-write-wins and siblings to CRDTs, operational transformation and the sync engines used by collaborative editors.

CRDT counters

A conflict-free replicated data type lets replicas update independently and still converge. A G-counter keeps one slot per replica and merges by taking the maximum, so no increment is lost or counted twice.

OR-sets and add-wins

Sets are where naive merges break. An observed-remove set tags every add with a unique ID, so a remove only deletes what its replica has seen. A concurrent add survives.

Text editing: OT vs sequence CRDTs

When two people type at the same spot, position-based edits diverge. Operational transformation fixes positions on a central server, and sequence CRDTs give each character an ID so no server is needed.

Design a collaborative editor

A design exercise for a rich-text editor where many people type at once, see each other's cursors, work offline and keep history. Local CRDT updates, a sync server with an update log, and periodic snapshots.

Check-then-act races

Why reading a seat's status and then writing it back can sell the same seat twice, and how one conditional UPDATE turns the check and the claim into a single atomic step.

Quorum reads and writes

With N replicas, a write succeeds after W of them acknowledge, and a read waits for R. When R + W is greater than N, every read overlaps every successful write in at least one replica.

Sloppy quorums and hinted handoff

When a home replica is down, Dynamo writes to a stand-in node and attaches a hint naming the intended owner. Writes keep working, and the hint is handed back when the owner returns.

Merkle anti-entropy and read repair

Two ways replicas heal after a missed write. Read repair fixes keys when someone reads them. Merkle-tree anti-entropy finds and fixes cold keys by comparing hashes, sending only the ranges that differ.

Dynamo vector clocks

Dynamo tags each version with a vector clock, a list of counters per writer. If one clock dominates another, the newer version wins. If neither dominates, the writes were concurrent, and both are kept as siblings.

Architecture patterns

Window counting limiters

Fixed windows, sliding logs and sliding counters, and the boundary burst that makes the simplest one leak.

Token bucket

A bucket that holds tokens and refills over time, giving you two dials for a sustained rate and a burst size. The most common shape for API limits.

Rate limiting

Why systems cap how much work a client can send, which algorithm counts, where the limit lives and what the client is told. Start here for the rate-limiting map.

Distributed limiter on Redis

How many servers share one rate-limit count without races, using an atomic Redis Lua script, one clock, expiring keys and a deliberate failure mode.

Local plus global limits

How high-volume limiters avoid a Redis call per request by deciding locally and syncing every few hundred milliseconds, and what overshoot that costs, worked through a 1 million requests per second design.

Limit responses and quotas

What a rate-limited client should receive (429, Retry-After and budget headers), and why a quota is a different, durable kind of limit.

Token limits for LLM APIs

Why LLM APIs limit tokens per minute as well as requests, and how a gateway reserves the worst-case output up front and refunds the unused part.

Why duplicates happen

Retries at every layer and at-least-once queues mean the same request can arrive twice. Learn where duplicates come from, and why the receiver must tolerate them.

Naturally idempotent operations

Some operations are safe to repeat by design, such as absolute sets, upserts and conditional writes. Learn to spot them, and to redesign the rest before you add keys.

Idempotency and retries

Making operations safe to retry, with idempotency keys, deduplicating consumers and jittered backoff, so at-least-once delivery never means a double charge.

Idempotency keys

For operations that can't be made idempotent by design, a client-chosen key lets the server recognize a retry and replay the first answer. Learn the protocol and its failure cases.

Deduplicating in message consumers

A consumer can see the same message twice. Learn three ways to make a duplicate harmless, and how the inbox and outbox patterns fit together.

Retry policy and backoff with jitter

Idempotency makes retries safe, and a retry policy makes them useful. Learn which errors to retry, how to space attempts with exponential backoff and full jitter, and when to stop.

Retry budgets and retry amplification

Retries at several layers multiply load exactly when a system is weakest. Learn the multiplication, and how to cap it with one retry layer and a retry budget.

Cascading failures

How one slow dependency exhausts threads, then turns into a full outage through retries and overload, with the arithmetic that shows why.

Timeouts and deadlines

A timeout caps how long one call waits, and a deadline caps the whole request. Set timeouts from latency data, and pass the remaining budget down every hop.

Retry budgets and hedging

When a retry helps and when it multiplies load, how to space retries with jittered backoff and a budget, and how hedged requests cut tail latency for reads.

Resilience patterns

The toolkit for keeping one failure from becoming an outage, from timeouts and retries to circuit breakers, bulkheads, load shedding and cells. Start here for the resilience map.

Circuit breakers

A circuit breaker watches a dependency's recent failures, trips to fail fast when the failure rate is high, and probes again after a cool-down.

Bulkheads and isolation

How giving each dependency or workload its own bounded pool of resources stops one failing part from taking the whole service down.

Load shedding and degradation

How to reject excess work early and by priority, so the work you accept still finishes on time, and how to serve a worse but usable experience instead of an error.

Limiting blast radius

Architecture that keeps a failure small by construction, using static stability, cells and shuffle sharding, with the arithmetic behind each.

Monolith vs microservices

The spectrum from monolith to modular monolith to microservices, what each one costs, and how Shopify and Segment reached opposite answers.

Conway's law and team shape

Why a system tends to copy the communication structure of the team that builds it, and how microservices mostly help by letting teams ship on their own schedules.

Bounded contexts

How to find service and module boundaries by splitting along business meaning, and why splitting by technical layer almost always fails.

Monoliths and microservices

When to split a system, how to draw service boundaries, and the operational cost that microservices add.

Service communication: synchronous vs asynchronous

How synchronous calls and asynchronous events couple availability differently, how failures multiply down a call chain, and how a distributed monolith forms.

Database per service

Why each service should own its data, what a shared database costs, and how to live with duplicated data, no cross-service joins and no cross-service transactions.

The operational tax of microservices

What a team needs in place before it splits into services, including discovery, tracing, per-service pipelines, contract tests, versioning, resilience and service identity.

Strangler fig migration

How to move from a monolith to services one capability at a time, with a router in front, and why "monolith first" is the default advice.

Events, commands and notifications

Commands ask one receiver to act, while events announce facts that anyone can react to. Learn the difference in coupling, and how thin and fat events trade consistency for independence.

Change data capture

CDC turns committed database changes into events by reading the database's own replication log, which avoids dual writes. Learn how it works, and why an outbox table usually sits in front of it.

Event-driven architecture, CQRS and event sourcing

Building systems around events, from change data capture to event sourcing and read models, and knowing when it simplifies a system versus creating a distributed mess.

Event sourcing

Store the events as the source of truth and derive the current state by replaying them. Learn the gains in audit and replay, and the costs of snapshots, schema changes and querying.

CQRS and read models

Command Query Responsibility Segregation uses separate models for writing and reading. Learn how read models are built from events, the eventual consistency they bring, and how to hide that lag from users.

Pitfalls of event-driven systems

Event-driven systems fail in predictable ways, including hard debugging, ordering gaps, duplicates, event soup and personal data in immutable logs. Learn the fix for each before the first event ships.

Worked example for an order system on an event log

Design an order system around a durable event log, with sizing, consumers and failure scenarios. Learn how one event stream feeds search, analytics and email without coupling them.

Tenant isolation models

The spectrum from shared tables to a dedicated stack per tenant, what each model costs, and the enterprise requirements that push large customers toward more isolation.

Multi-tenant SaaS architecture

Designing one system for many customers, covering isolation models, tenant scoping, noisy neighbors, tenant-aware sharding and per-tenant cost.

Enforcing tenant scoping

How to make sure no query in a shared database returns another tenant's rows, using the tenant from login, automatic scoping, Postgres row-level security and leak tests.

Noisy neighbors

How one tenant's heavy load can slow everyone else, and the controls that contain it, from per-tenant limits and fair queuing to dedicated pools and shuffle sharding.

Moving tenants and sharding

How to shard a multi-tenant database by tenant, use logical shards and a directory, and move a large tenant online without downtime.

Cost per tenant

How to meter each tenant's use of shared resources, allocate the cloud bill back to tenants, and find the customers who cost more than they pay.

Why go multi-region

The three legitimate reasons to run in several regions (latency, regional disaster tolerance and residency), the costs they bring, and why multi-zone deployment comes first.

Active-passive, RPO and RTO

How a standby region takes over, what RPO and RTO mean, and how replication lag and human steps set the numbers.

Multi-region architecture

Active-passive, active-active and cell-based designs for latency, disaster tolerance and data residency, with the recovery, consistency and cost trade-offs each one carries.

Active-active and home regions

How every region can serve live traffic by giving each user or tenant a home region that owns its writes, plus the capacity math for surviving a region loss.

Globally consistent databases

Which data needs one consistent answer everywhere, how Spanner, CockroachDB, Aurora Global and DynamoDB pay for it, and why the cost lands on every write.

Cells and regional isolation

How regions share as little as possible, and how cells cap the damage from a bad deploy or config push to a small slice of users.

Failover testing

Why an untested failover plan is close to no plan, what the GitHub 2018 incident shows about failover going wrong, and the practices that keep failover working.

Scaling reads and writes

Read load is spread with copies, caches and CDNs. Write load is spread by splitting the data, queuing spikes and using stores built for fast appends. Each tool trades freshness or simplicity for capacity.

Hot keys and celebrities

One key or user can attract far more traffic than the rest. Sharding cannot spread a single key, so you split it, cache it, or change how updates fan out.

Contention and holds

When many clients change one scarce unit, such as a seat or a coupon, use a conditional update with a time-limited hold, plus a waiting room. Locks are simple to explain, but they fail under a sudden spike.

Multi-step consistency

When two systems must agree, such as inventory and payment, use a saga with compensating steps, an outbox for events, idempotency keys on every hop, and a reconciliation job. Two-phase commit everywhere is rarely the answer.

Realtime updates and long-running work

Clients need fresh state without polling hard. Push it over a live channel. Work that outlasts one request runs as a job, with a status resource and a webhook when it finishes.