Guide

Home

Syllabus

Every topic, section and page on one page. Use it to plan your path or to find where a topic lives.

Computer systems foundations

What actually happens between your code, the CPU, the OS, the network and the disk.

Machines and programs

  1. How code runsFrom source file to running instructions. Covers the CPU's loop, compilers, interpreters and JITs, branch prediction and the system call boundary.
  2. The CPU loopWhat a core does billions of times a second, why pipelining and out-of-order execution exist, and what instructions per cycle (IPC) tells you about your code.
  3. Compiled, interpreted and JITThe three ways a language turns source into machine work, who pays the cost and when, and why the same summing loop runs about 57 times faster in Go than in CPython.
  4. Branch predictionWhy a CPU guesses which way every if and loop goes, what a wrong guess costs, and why sorting an array can make an unrelated loop several times faster.
  5. System callsHow a program asks the kernel to read files, open sockets or allocate memory, what each request costs, and how strace shows every call a process makes.
  6. The memory hierarchyFrom registers to SSDs to the network, the latency numbers that dominate performance, and why the tier your data sits in matters more than the code you write.
  7. Reasoning with latency and bandwidthTurn the latency table into quick cost estimates, separate latency from bandwidth, and use Little's law to find how many requests must be in flight to use a link or memory fully.
  8. Cache lines and localityWhy caches move 64-byte blocks, how sequential access and hardware prefetching make arrays fast, and why walking a 2D array in the wrong order is six times slower.
  9. Array of structs vs struct of arraysHow the layout of records in memory decides how many useful bytes each cache line carries, why struct of arrays is faster for scans, and when the opposite layout wins.
  10. False sharing and NUMAWhy two threads writing to different variables in the same cache line slow each other down, and why memory on the far socket of a big server costs more to read.
  11. Stack, heap and allocationWhere values live in a running program, what escape analysis decides for you, and why allocating in a hot loop slows a program even when allocation itself is cheap.
  12. Memory management and garbage collectionHow languages decide where values live and when memory is given back, the trade-offs of manual, ownership, reference-counted and tracing approaches, and how to find leaks and size heaps safely.
  13. Four ways to free memoryManual free, ownership, reference counting and tracing garbage collection, what each one costs, and which languages use which.
  14. How tracing garbage collectors workMark-sweep, compaction and copying, the generational idea that most objects die young, and how concurrent marking with write barriers keeps programs running while the collector works.
  15. GC pauses and tuningThe throughput, latency and memory trade-off every collector makes, how a 100 ms pause every five seconds becomes a p99 problem, and the knobs that change it in Go and the JVM.
  16. Memory leaks and OOM killsWhat a leak looks like in a garbage-collected language, how to find one by diffing heap profiles, why process memory is bigger than the heap, and how a container gets OOM-killed.
  17. Binary, integers and overflowHow integers are stored as bits, why signed counters wrap around, and why byte order matters when you read raw binary.
  18. Numbers, text and encodingsHow integers, floats, text, timestamps and Base64 are really stored as bytes, and the production bugs each one causes. Start here for the data-representation map.
  19. Floating point and moneyWhy 0.1 + 0.2 is not 0.3, how floats lose precision, and why money must be stored as exact integers or decimals.
  20. Unicode and UTF-8How text becomes bytes. Code points, UTF-8 and UTF-16, why "length" has four answers, and how mojibake and normalization bugs happen.
  21. Base64 and URL encodingHow arbitrary bytes travel through text-only channels with Base64, and how URL encoding keeps query strings from breaking. Covers the injection bug that string concatenation causes.
  22. Timestamps and time zonesHow to store and send times so they stay correct across time zones and daylight saving, how leap seconds break things, and why durations need a monotonic clock.

Operating systems and concurrency

  1. File descriptorsThe small integer your program holds for each open file, socket or pipe, and why "too many open files" is either a low limit or a leak.
  2. Processes, threads and the kernelWhat the OS gives every program, and how to see it on a live Linux box. Start here for the process model.
  3. Process lifecycleHow a process is born with fork, takes on a new program with exec, and exits with a code. Why zombies build up when a parent never calls wait.
  4. Virtual memoryWhy every process sees its own private memory, how the kernel maps it to RAM in 4 KiB pages, why ps shows two memory numbers, and what triggers the OOM killer.
  5. CPU schedulingHow the kernel decides which thread runs on which core, why extra threads cost time, and why a container with low average CPU can still stall.
  6. Signals and graceful shutdownHow the kernel tells a process to stop, why SIGTERM can be caught and SIGKILL cannot, and how a service drains work before it exits. Covers the PID 1 trap in containers.
  7. Race conditionsA race condition happens when the result depends on the order in which threads run. How a read-modify-write loses an update, and why check-then-act bugs are so common.
  8. ConcurrencyWhy shared state breaks when two threads touch it at once, and the tools that fix it, from locks and semaphores to atomics, message passing and database constraints.
  9. Mutexes, semaphores and condition variablesThe three basic blocking primitives. A mutex gives exclusive access, a semaphore counts available resources, and a condition variable waits for a state change.
  10. Atomics and memory orderingAtomic operations change one word in a single indivisible step, compare-and-swap builds lock-free updates, and memory ordering decides what other threads may see.
  11. Deadlock and livelockDeadlock is threads waiting on each other forever. The four Coffman conditions that must all hold, the fixes that break one of them, and how livelock differs.
  12. Shared state without locksDesigns that avoid shared mutable state, such as message passing, database constraints and fencing tokens, and why they hold up better across machines.
  13. Blocking I/O and epollHow a thread blocks on a socket, why select and poll slow down at scale, and how epoll and kqueue report only the sockets that are ready.
  14. Async I/OWhy waiting on the network should not hold a thread, how event loops and async functions handle thousands of connections, and when goroutines or thread pools fit better.
  15. Event loopsAn event loop runs a queue of ready callbacks and waits on the kernel for more. Why one slow callback stalls everything, and how to keep the loop responsive.
  16. Async and await state machinesWhat async and await compile down to. Each await is a suspension point, the function becomes a state machine, and function color is why async spreads through a codebase.
  17. Goroutines and green threadsLightweight threads scheduled by the runtime instead of the OS. How goroutines use a netpoller, why M:N scheduling scales, and the trade-off against async/await.
  18. Backpressure and bounded queuesBackpressure lets a slow consumer push back on a fast producer. Bounded queues make the limit explicit, and each overflow policy trades latency against lost work.
  19. File systems and journalingHow a file system maps names to inodes, why hard links share one inode, and how a journal keeps metadata consistent after a crash.
  20. Page cache and fsyncWhy a successful write only reaches RAM, how fsync and fdatasync force data to disk, when O_DIRECT helps, and why a failed fsync cannot be retried.
  21. Files and storageHow file systems map names to inodes, how journaling and the page cache affect durability, and how the storage devices underneath behave, from SATA disks and SSDs to cloud volumes.
  22. Durable file writesThe atomic replace pattern for updating a file so a crash leaves either the old or the new version, plus how torn writes are detected with checksums.
  23. Storage devices and SSDsHow HDDs, SATA SSDs and NVMe drives differ in latency and random I/O, why flash must erase before writing, and how write amplification and endurance limit SSDs.
  24. Cloud storage and RAIDHow RAID trades capacity for redundancy, how erasure coding does it at scale, and what cloud block volumes, instance stores and object storage each promise.

Networking

  1. Packets, layers and addressesHow data is wrapped in layered headers, what the MTU and MSS limit, and how IPv4 addresses, CIDR blocks, private ranges and NAT decide what can reach what.
  2. Networking: IP, TCP and UDPPackets, addresses, ports, and the handshakes and congestion control that decide how fast your bytes arrive. Explains why a new connection is slow and how to check it on the wire.
  3. TCP handshake and reliabilityHow TCP sets up a connection with a three-way handshake, numbers every byte, recovers lost packets, and closes cleanly, along with the costs each step adds.
  4. Congestion control and BDPWhy a new TCP connection starts slow, how slow start works with a 100 KB example, how CUBIC and BBR differ, and how the bandwidth-delay product tells you how much data must be in flight.
  5. TCP latency traps and UDPWhy TCP's ordered stream causes head-of-line blocking, how Nagle's algorithm and delayed ACKs combine into a 40 ms bug, the fix with TCP_NODELAY, and when UDP is the better transport.
  6. Network toolsHow to check the claims in the networking pages with ping, mtr, ss and tcpdump, including how to read a handshake in a capture and what each timestamp means.
  7. DNS resolution walkTrace a lookup from the stub resolver through recursive resolvers, root, TLD and authoritative servers, and see where each cache sits along the way.
  8. DNSHow names become addresses, which record types matter, how caching and TTLs shape change speed, and why DNS failures look like outages of everything else.
  9. TTLs and cachingHow TTLs set the speed of DNS changes, how to lower them ahead of a migration, and why some resolvers ignore small values.
  10. DNS traffic steering and anycastHow DNS answers can route users by weight, location, latency or health, and how anycast sends the same IP address to the nearest of many servers.
  11. DNS outages and ndotsWhy DNS failures take down unrelated services at once, what the 2016 Dyn and 2021 Facebook outages teach, and how the Kubernetes ndots setting multiplies lookups.
  12. DNS securityHow DNS answers can be forged, what DNSSEC does and does not protect, and how DNS over HTTPS and TLS hide queries from networks in between.
  13. Requests, methods and status codesWhat an HTTP request and response look like on the wire, which methods are safe and idempotent, and which status codes tell clients and proxies what to do next.
  14. HTTP/1.1, HTTP/2 and HTTP/3The protocol of the web. Separates HTTP's meaning (methods, status codes, caching) from its wire format, and shows how each version fixes the latency problem of the one before.
  15. HTTP/1.1 and HTTP/2 multiplexingWhy HTTP/1.1 connections handle one request at a time, how browsers and developers worked around it, and how HTTP/2 multiplexes streams over one connection while still stalling on TCP loss.
  16. HTTP/3 and QUICHow HTTP/3 runs on QUIC over UDP to recover lost packets per stream, merge transport and TLS setup into one round trip, and keep connections alive when the phone changes networks.
  17. HTTP cachingHow Cache-Control, ETag and Vary let browsers and CDNs skip work, with a standard pattern of fingerprinted assets for a year and revalidated HTML.
  18. Cookies, CORS and streamingHow cookies carry state in the browser, why CORS preflights cost a round trip, how compression and chunked responses shape delivery, and when to pick server-sent events, WebSockets or gRPC.
  19. What TLS guaranteesThe three things TLS does for every HTTPS connection (confidentiality, integrity and authentication), where it sits in the stack, and what it does not hide.
  20. TLS and certificatesHow HTTPS keeps data private and proves the server's identity, how the TLS 1.3 handshake works, and why expired certificates keep causing outages. Start here for TLS.
  21. The TLS 1.3 handshakeHow client and server agree on secret keys in one round trip, why the keys are thrown away afterwards, and what 0-RTT resumption costs in replay risk.
  22. Certificates and chains of trustWhat an X.509 certificate says, how a leaf certificate links to a trusted root through intermediates, why a missing intermediate breaks some clients, and how Let's Encrypt and Certificate Transparency fit in.
  23. Certificate expiry and renewalWhy certificates expire on purpose, why expired ones take sites down, and the playbook for automated renewal, independent monitoring and a full inventory. Includes a check you can run in a minute.
  24. Terminating TLS and mTLSWhere an encrypted connection ends and plaintext begins, the three common placements, and how mutual TLS with short-lived workload identities protects traffic between services.
  25. Life of a requestEvery step between typing a URL and seeing a page, with a latency budget for each hop, why the second request is faster, and a method for finding where a slow request loses its time.
  26. Latency budget and round tripsHow to put numbers on a cold request from Mumbai to Virginia, why round trips dominate for distant users, and how a CDN cuts the time to first byte from about 890 ms to about 62 ms on a cache hit.
  27. Connection reuseWhy the second request to the same server is several times faster than the first, where connection reuse appears across browsers, CDNs, services and databases, and how to configure a Go HTTP client to pool connections correctly.
  28. Measuring a requestHow to read a browser waterfall, send Server-Timing headers, follow a request with distributed traces, and break down the 60 ms of server time for a product page.
  29. Debugging a slow endpointA hop-by-hop method for a p99 regression. Start with whether the median moved, pull traces of slow requests, and test one hypothesis per hop with the matching tool.

Data structures and storage engines

  1. Arrays and memory layoutWhy contiguous arrays are fast on real hardware, how cache lines and prefetching reward sequential access, and how a fixed-size ring buffer handles streams with bounded memory.
  2. Hash mapsHow hash maps map keys to buckets, how load factor and collisions set their speed, and why adversarial keys can turn O(1) lookups into O(n).
  3. Data structures in practiceChoosing data structures by memory layout, access pattern and the operations that matter, with the common ones for arrays, hash maps, heaps, trees, caches and graphs.
  4. Heaps and top-kHow a binary heap keeps the smallest or largest item at the top, and how a heap of size k finds the top k in one pass without sorting everything.
  5. Sorted trees, skip lists and triesThe structures that keep keys in order for range queries, the red-black and skip list options for in-memory sorting, and tries for prefix search.
  6. LRU cachesHow an LRU cache evicts the least recently used item in O(1) by pairing a hash map with a doubly linked list, and why a plain array or sorted list is too slow.
  7. Graphs and topological sortHow to represent a graph with adjacency lists, and how Kahn's algorithm orders tasks with dependencies, and detects a cycle when no valid order exists.
  8. Bloom filtersA compact set-membership test that never misses an item it has seen, but sometimes says yes to one it has not. How to size it for a target error rate.
  9. Probabilistic data structuresCompact structures that answer membership, distinct-count, frequency, sampling and similarity questions with a small, bounded error. Start here for the family and how to choose one.
  10. HyperLogLogHow to count distinct items in a fixed 12 KB with about 0.8% error, using the longest run of leading zeros in hashed values. Why these sketches merge cleanly.
  11. Count-min sketchA fixed-size table that estimates how often each item appears, never underestimates, and finds trending items. Covers the error bound and how to size it.
  12. Reservoir samplingHow to pick a fair random sample of k items from a stream when you do not know its length, in one pass and k slots of memory. Includes why every item has the same chance.
  13. MinHash and LSHHow to estimate how similar two sets are from tiny signatures, and how locality-sensitive hashing finds near-duplicate pairs without comparing every pair.
  14. Pages and row layoutA database page is a fixed-size block with a header, slots and rows packed from both ends. How that layout drives row IDs, updates and wasted space.
  15. Storage engines: pages, B-trees and the WALHow a relational database stores rows in fixed-size pages, finds them in a B+tree, caches them in a buffer pool, and survives crashes with a write-ahead log.
  16. B+tree lookupsHow a B+tree finds a key in a few page reads, why its fanout is large, how page splits happen, and why random keys hurt inserts.
  17. Buffer poolThe buffer pool caches database pages in RAM. How eviction chooses which page to drop, why sequential scans can wipe out the cache, and how midpoint insertion protects hot pages.
  18. Write-ahead logThe write-ahead log records every change before the page is written. How commits become durable, how recovery replays the log, and what group commit and checkpoints do.
  19. Composite and covering indexesHow column order in a multi-column index decides which queries it serves, and how including extra columns lets a query skip the table entirely.
  20. Storage engines II: LSM treesWrite-optimized storage built from memtables, sorted immutable files and background compaction. Covers how it reads, deletes and pays for its writes, and how it compares with a B-tree.
  21. Writes, memtables and SSTablesHow an LSM tree turns every write into a log append and a memtable insert, flushes sorted immutable files, and never overwrites data on disk.
  22. Reads, deletes and Bloom filtersHow an LSM tree checks several places to find the newest value, how Bloom filters keep that close to one disk read, why range scans are the weak spot, and how tombstones make deletes hard, including Cassandra's zombie rows.
  23. Compaction strategiesWhy LSM trees must merge their files, how leveled compaction keeps few sorted runs at high write cost, and how size-tiered compaction keeps writes cheap at the price of more files and more disk space.
  24. Write amplificationThe read, write and space costs every storage engine trades off, a worked example of how leveled compaction multiplies one application write into about 43 device writes, and what that does to SSD endurance.
  25. Transaction anomaliesThe concurrency bugs a database can allow when transactions overlap, shown as timelines of two transactions with the SQL that causes each one: dirty read, non-repeatable read, phantom read and lost update.
  26. Transactions, MVCC and isolation levelsWhat ACID guarantees on one database node, which concurrency bugs each isolation level still allows, how MVCC gives snapshots without blocking, and the code patterns that keep concurrent writes correct. Start here for transactions.
  27. Isolation levelsWhat each isolation level promises, which anomalies it still allows, and how Postgres and MySQL differ in practice, including write skew, which snapshot isolation does not prevent.
  28. MVCC and snapshotsHow multi-version concurrency control lets readers and writers avoid blocking each other, how Postgres row versions and snapshots work, and why vacuum and long-running transactions matter.
  29. Locking and concurrency controlHow row locks, SELECT FOR UPDATE, SKIP LOCKED, deadlocks, version columns and advisory locks make writers wait safely, with a job queue and an optimistic update as worked examples.
  30. Concurrency patterns that prevent racesFive habits that stop most concurrency bugs in application code, with SQL for atomic updates, unique and exclusion constraints, and a Python retry loop for serializable transactions.
  31. Query plans and executionHow a database turns SQL text into a tree of operators, picks the cheapest one, and runs it row by row. Introduces the query plan and the difference between EXPLAIN and EXPLAIN ANALYZE.
  32. How a SQL query executesHow a database plans and runs a query, which access paths and join algorithms it chooses between, and how to read EXPLAIN ANALYZE when a query is slow. Start here for query performance.
  33. Access paths and selectivityHow a database finds a table's rows, either by reading every page or by using an index. Why the planner sometimes skips an index you expected, and what selectivity has to do with it.
  34. Join algorithmsThe three ways a database joins two inputs (nested loop, hash join and merge join), what each costs, and when the planner picks each one.
  35. Statistics and cardinality estimationHow the planner estimates how many rows each step produces, why those estimates go wrong for correlated columns, and how to refresh statistics before rewriting a query.
  36. Reading EXPLAIN ANALYZEHow to read a real query plan bottom up, find the step that costs the time, and fix it with a matching index. Includes the common query patterns that cause slow plans.
  37. Text vs binary formatsWhat serialization is, why a service's data format sets its size and cost, and how JSON compares with Protobuf, Avro and Parquet on the same record.
  38. Serialization, schemas and compressionHow data becomes bytes for the wire and for disk, how formats like JSON, Protobuf and Avro differ, how to change a schema safely, and when compression pays. Start here for data formats.
  39. Binary schemas, Protobuf and AvroHow Protobuf uses permanent field numbers and Avro uses writer and reader schemas to read compact binary data, and when to choose each.
  40. Schema evolutionHow to change a record's shape without breaking the producers and consumers that use it. Backward and forward compatibility, safe Protobuf and Avro changes, and schema registry modes.
  41. Compression tradeoffsWhen compressing a payload saves more than it costs, how the common codecs trade size for CPU, and how to measure the trade on your own data.
  42. Serialization pitfallsThe production bugs that come from encoded data. Large integers in JSON, mixed timestamp units, enums, null versus absent, money as floats, and checksums that are not security.

Engineer's toolkit

  1. Hash functionsA hash turns any input into a fixed-size fingerprint. A secure hash is hard to reverse and hard to collide, and MD5 and SHA-1 no longer meet that bar.
  2. Secure randomness and tokensWhere keys, nonces, salts and session tokens must come from, why Math.random is unsafe, and how many bits a token needs.
  3. Cryptography for engineersThe few building blocks you need for everyday security decisions, which question each one answers, and where teams go wrong. Start here for the map.
  4. HMAC and webhook signingA keyed hash proves a message came from a key holder and was not changed. It protects webhooks, signed URLs and TOTP codes.
  5. AEAD and envelope encryptionAES-GCM encrypts and authenticates in one step, nonces must never repeat under one key, and envelope encryption keeps the master key inside a KMS.
  6. Public-key crypto and signaturesKey pairs let strangers agree on a secret and let anyone verify a signature. Why elliptic curves are the default, what ECDHE gives you, and when a signature beats an HMAC.
  7. Password hashingPasswords need slow, salted, memory-hard hashes. How the guessing math works, which algorithm to pick, and how to tune the cost for your login servers.
  8. Sessions and cookiesHow a browser login stays alive. Cookie attributes that matter, session fixation, CSRF, and why server-side sessions are easier to revoke than signed tokens.
  9. Authentication and authorizationAuthentication proves who you are, authorization decides what you can do, and service-to-service calls need their own identity. The map for the rest of this topic.
  10. JWTs and token validationA JWT is a signed claim set, not a secret. How to validate one safely, the alg confusion and none attacks, and when a JWT is the wrong tool.
  11. OAuth and OpenID ConnectOAuth 2.0 delegates access to a third-party app. OpenID Connect adds a login identity on top. The authorization code flow with PKCE is the safe default.
  12. Authorization models and IDORCompare RBAC, ABAC and relationship-based access, and learn how insecure direct object references let users read each other's data.
  13. MFA and passkeysHow TOTP codes work, why SMS is the weakest second factor, and how passkeys resist phishing by binding each login to a single website.
  14. Git's object storeThe four object types Git stores, how a hash of the contents becomes the object's ID, and why that gives Git deduplication, integrity and cheap comparisons.
  15. Refs, HEAD and the reflogBranches, tags and HEAD are small files that point at commits, and the reflog records every move, which is why almost any committed work can be recovered.
  16. Git internals and workflowGit as a content-addressed object store plus a graph of commits and movable pointers, so merges, rebases, bisects and recoveries stop feeling risky, along with the team workflows that go with them.
  17. Merge, rebase and squashThe three ways to combine a branch into main, what each one does to the commit graph, how to resolve conflicts without guessing, and the one rule about rewriting shared history.
  18. Editing history and recovering workHow interactive rebase, fixup commits and backup branches make your own history clean, and the recovery steps for a botched reset, rebase or deleted branch.
  19. Bisect regressionsHow git bisect finds the commit that introduced a bug in about log2(N) steps, how to automate it with a script, and why small, buildable commits make it work.
  20. Team workflowsTrunk-based development versus GitFlow, why small PRs and stacked diffs speed up review, and how monorepos and polyrepos trade tooling cost against coordination cost.
  21. Latency and percentilesHow to describe how long things take. Why averages hide slow requests, what p50, p99 and the tail mean, and how fan-out makes a rare slow call common.
  22. Performance and profilingHow to find where time goes before you optimize. Percentiles, two predictive laws, sampling profilers, off-CPU waiting and honest benchmarks, with a worked investigation of a slow endpoint. Start here for the performance toolkit.
  23. Amdahl's and Little's lawsTwo formulas that predict the payoff of a speedup and the concurrency a system needs before you change any code. Includes why a 16-core job is not 16 times faster.
  24. Flame graphs and CPU profilingHow sampling profilers find where CPU time goes, how to read a flame graph, and how allocation profiles reveal GC pressure.
  25. Off-CPU timeWhy a service at 10% CPU can still have slow requests, how to measure the time spent waiting, and which tools show it.
  26. Benchmarking without lyingHow micro-benchmarks mislead you, through warmup, optimized-away code, constant folding, noise and fake data, and how to write one that tells the truth.

System design

How large systems stay fast, available and correct, from building blocks to full case studies.

Fundamentals

  1. Napkin constantsThe handful of numbers worth memorizing so you can estimate scale in your head, from seconds per day to what one server can handle.
  2. Latency numbers every engineer should knowA reference table of how long memory, disk and network operations take, and the ratios that decide where a design can afford to wait.
  3. Back-of-the-envelope estimatesTurn users and features into QPS, storage and bandwidth numbers in two minutes, and use them to decide what the design actually needs.
  4. The estimation chainThe five steps that turn users and behavior into queries per second, storage, bandwidth and server count, with the multipliers you should say out loud.
  5. Worked example for a Twitter-like appA full estimate for a timeline-based social app, from requests per second to egress cost and server count, with each number tied to a design decision.
  6. Read/write ratio and the one-machine checkTwo checks to run after the numbers. The read:write ratio tells you where to spend effort, and the one-machine check asks whether you need a distributed system at all.
  7. Estimation practiceThree practice problems, photo sharing, chat and analytics events, each with its assumptions, arithmetic and the design conclusion it forces.
  8. Vertical versus horizontal scalingScaling up with a bigger machine versus scaling out with more machines, what each costs, and when scaling up stops being the right answer.
  9. Stateless servicesA stateless service can handle any request on any instance. Learn where the state goes instead, and the login bug that appears when a service keeps sessions in memory.
  10. Scaling fundamentalsVertical versus horizontal scaling, stateless services, the usual order in which a growing system adds parts, and how to find the real bottleneck. Start here for the scaling map.
  11. Redirect status codes: 301 vs 302A redirect's status code tells browsers and CDNs whether they may keep the answer, which decides whether your server ever sees the next click.
  12. The canonical scaling pathThe usual sequence in which a successful web app adds parts, from one server to queues, shards and regions, with the symptom that triggers each step.
  13. The scale cubeThree independent ways to split a system, by cloning, by function and by data, and how to name the next step in a design review.
  14. Finding the bottleneckHow Amdahl's law, Little's law and utilization tell you which resource limits a system and how much a fix will actually help. The database is usually the answer.
  15. Growing a marketplaceA worked plan for a small marketplace that expects 10 times its traffic in a year, step by step with real numbers, and why it needs no sharding.
  16. Nines and downtimeAvailability is the share of time or requests that work. Each extra nine cuts allowed downtime tenfold, and the last few nines need automation, not heroics.
  17. MTBF and MTTRAvailability depends on how often things break and how long each break lasts. Cutting recovery time often adds more nines than preventing failures.
  18. Percentiles and tail latencyLatency is a spread of values, not one number. Averages hide the slow requests that users feel, so describe latency with p50, p99 and p99.9.
  19. Availability, latency and the tailNines, percentiles and the math of composing services. Why p99 matters more than the average, how fan-out amplifies the tail, and how to design for both uptime and speed.
  20. Composing availabilityA request that needs every dependency multiplies their availabilities, which lowers the ceiling. Redundancy adds nines only when the replicas fail independently.
  21. Tail latency amplificationA request that fans out to many backends waits for the slowest one. With enough fan-out, a rare slow backend becomes the normal response time.
  22. Taming the tailYou can't stop every server from being slow. Set timeouts from measured latency, send a hedged second request when the first is late, and return partial results on time.
  23. Latency budgetsA latency budget splits one end-to-end target across every hop, so each team knows its share. Parallel calls cost the slowest one, and sequential calls add up.
  24. API stylesREST, gRPC and GraphQL can answer the same request. Pick one by asking who calls the API and what they need from it.
  25. Safe methods and idempotencyEach HTTP method makes a promise about side effects. Knowing which methods are safe to repeat tells you when a retry is harmless and when it could charge a card twice.
  26. API designAn API is a contract that outlives the code behind it. Design the styles, methods, pagination, errors and async flows so changes stay additive.
  27. Cursor paginationOffset pagination skips rows and gets slower and less stable as you go deeper. Cursor pagination asks for the rows after the last one seen, so every page costs the same.
  28. API versioningThe best version is the one you never ship. Make additive changes, make clients tolerant of new fields, and version only when you must break the contract.
  29. Structured errorsA good error response says what went wrong, whether to retry and how to fix it, in a form code can read. Use status codes, a stable error code and an ETag for concurrent edits.
  30. Webhooks and long-running operationsWork that takes minutes should return 202 at once and finish in the background. Clients then poll an operation or receive a signed webhook when it is done.
  31. Little's lawA one-line formula that links how many items are in a system, how fast they arrive, and how long each one stays. It sizes queues, worker pools and servers.

Data building blocks

  1. Access patternsAn access pattern is one concrete query your system runs, with its lookup shape, rate and latency target. Write the list first, and let the choice of store follow from it.
  2. Databases and data modelingChoose between relational, document, wide-column, key-value, graph and analytical stores by listing your access patterns first, then model the data to fit them.
  3. Indexing strategyAn index is a sorted copy of some columns that lets the database jump to matching rows. Column order, covering and partial indexes decide how useful it is, and every index slows every write.
  4. Query-first modelingKey-value and wide-column stores have no joins, so you name the partition key for every query before you store anything. DynamoDB and Cassandra both work this way.
  5. Specialized data storesGraph, time-series, search, vector and columnar stores each win one access pattern that general-purpose databases handle poorly. Columnar storage explains why analytics reads far less data.
  6. An order system, modeled two waysModel the same order system once in Postgres and once in DynamoDB, using the same access patterns. The comparison shows what you gain and what you give up with each.
  7. Single-leader replicationOne node accepts every write and streams its log to followers. The choice between synchronous and asynchronous commits decides whether a crash can lose an acknowledged write.
  8. How changes are shippedThe four ways a database describes a change to its replicas, statement, physical, logical and change data capture, and the failure each one invites.
  9. ReplicationKeeping copies of the same data on several machines, the three topologies that databases use, and the anomalies that replication lag causes. Start here for the replication map.
  10. Failover and split brainThe four steps of promoting a follower after the leader fails, the data that asynchronous replication can lose, and how fencing prevents two leaders from writing at once.
  11. Replication lag anomaliesWhy reads from a lagging replica can show the past, the three user-visible anomalies this causes, and the standard fix for each.
  12. Multi-leader replicationWhen several nodes accept writes at once, why that needs a conflict plan, and the four ways to settle a conflict, from avoiding it to merging with domain rules.
  13. Leaderless quorumsHow Dynamo-style stores accept writes with no leader, using W and R quorums, sloppy quorums with hinted handoff, and repair with read repair and Merkle trees.
  14. Range and hash partitioningRange partitioning gives each shard a block of neighboring keys, which keeps scans cheap but can hot-spot. Hash partitioning spreads writes evenly but scatters neighbors.
  15. Sharding and partitioningSplit data across machines by choosing a shard key, hashing or ranging it, routing requests, handling hot spots, resharding without downtime and generating IDs that stay unique.
  16. Choosing a shard keyA good shard key has high cardinality, even traffic, query locality and stability. Choose it from your most common queries, and plan for the single hot value that breaks even a good key.
  17. Secondary indexes when shardedWhen data is sharded by its primary key, queries on other fields either visit every shard (local index) or use an index that is itself sharded (global index), with different costs for reads and writes.
  18. Resharding without downtimeTwo ways to add capacity to a sharded system, and a dual-write playbook for moving data while traffic keeps flowing. Each step stays reversible until the final cleanup.
  19. Cache layersA request can be served from a cache at many layers, from the browser to the database. The closer a cache sits to the user, the more load it saves and the harder it is to invalidate.
  20. Cache patternsA cache pattern decides who loads data into the cache and when writes reach the database. Cache-aside is the usual choice, because the cache is optional and the app still works without it.
  21. Hit rate and sizingHit rate sets how much traffic reaches the database. Going from 90% to 99% cuts database load tenfold, so size the cache for the working set and measure hit rate as a capacity metric.
  22. Redis specificsRedis is the default shared cache, and it stores far more than strings. Know its data structures, its single-threaded commands, its eviction settings and how cluster mode splits keys.
  23. CachingWhere to put a cache, how to read and write through it, how to size it from hit-rate math, and how to keep it from serving stale data or melting down.
  24. Cache invalidationA cache learns that its data is stale through TTLs, deletes on write, versioned keys or change-data-capture. Each gives bounded staleness, not strong consistency, so choose by how stale you can tolerate.
  25. Cache stampedeWhen a hot key expires, every request misses at once and runs the same expensive query. Fix it by letting one request recompute, refreshing early, or serving slightly stale data.
  26. Cache failure modesCaches also fail in three other ways. One key can get too much traffic, requests can ask for keys that don't exist, and an empty cache can send every read to the database. Each has a specific fix.
  27. Memcache at FacebookFacebook's 2013 memcache design uses leases, regional pools, commit-log invalidation and gutter pools. It is a catalog of what goes wrong when a cache becomes load-bearing.
  28. The object storage modelObject storage keeps whole blobs under flat keys behind an HTTP API. Learn that model first, and the rest of the storage design follows from it.
  29. Durability and erasure codingObject stores reach eleven nines of durability with redundancy and fast repair. Erasure coding survives more failures than three full copies, at under half the storage cost.
  30. Object storageStore files and media in S3-style object stores. Keep bytes out of your app servers and databases, and design uploads, metadata and costs around that split.
  31. Presigned and multipart uploadsLet clients upload straight to object storage with a signed, time-limited URL, and split big files into parts so one dropped connection costs one part, not the whole file.
  32. Metadata and object consistencyKeep bytes in object storage and queryable facts in a database. Write intent first, confirm after the upload, and clean up what a crash leaves behind.
  33. Storage classes and egress costObject stores price storage by how often you read it. Move cold data to cheaper classes with lifecycle rules, and serve hot content through a CDN, because data leaving the cloud is often the largest bill.
  34. File sync designA Dropbox-style sync client stores each file as hashed chunks, uploads only the chunks the server lacks, and commits versions with a base-version check so concurrent edits never silently overwrite.
  35. Inverted indexMap every word to the sorted list of documents that contain it, so a search becomes a few list lookups and a merge instead of a scan of every document.
  36. Text analysis: tokens, stems and synonymsBefore a word enters the index, an analyzer splits, lowercases, filters and stems it. The same analyzer must run on queries, and changing it means reindexing everything.
  37. Search systemsHow inverted indexes, analyzers and BM25 scoring work, and how to run Elasticsearch-style search at scale with sharding, indexing pipelines and hybrid vector retrieval.
  38. BM25 rankingHow BM25 scores a document for a query, using term frequency with saturation, a rare-word weight and length normalization, worked by hand and in code.
  39. Distributed search: query then fetchHow a search index split into shards answers one query. Each shard scores its own documents, the coordinator merges the top hits, and then only the winning documents are fetched.
  40. Search indexing pipelineHow a search index stays in sync with its source database using change data capture, idempotent versioned writes and zero-downtime reindexing behind an alias.
  41. Hybrid search with reciprocal rank fusionCombine keyword ranking with vector similarity. Reciprocal rank fusion merges the two lists using only ranks, so you never compare their raw scores.
  42. MapReduceA batch model with two user functions. Map emits key-value pairs, the framework shuffles them so each key lands on one reducer, and reduce combines the values. Failed tasks are simply rerun.
  43. Spark and shufflesSpark runs a pipeline as a graph of steps and keeps data in memory where it can. Narrow steps stay on one machine, wide steps need a shuffle, and the shuffle usually decides the cost.
  44. Batch and stream processingHow to move and transform large data sets reliably. Batch jobs run over bounded data, streams run over events as they arrive, and the same ideas of time, state and idempotent writes apply to both.
  45. Event time, windows and watermarksEvery event has the time it happened and the time the system saw it. Aggregating by event time gives reproducible answers. A watermark says when a window is complete, and late events are handled after that.
  46. Exactly-once in streamsA stream processor restores its state and rewinds its input after a crash. The state counts each event once, but the output can repeat. End-to-end exactly-once needs an idempotent or transactional sink.
  47. Lambda vs KappaTwo ways to keep analytics fast and correct. Lambda runs a batch layer and a speed layer with duplicated logic. Kappa uses one streaming pipeline and replays the log when it needs to reprocess.
  48. Design an ad-click aggregation pipelineDesign exercise. Count clicks per ad per minute within about a minute, pace budgets, and bill exactly. A streaming path gives freshness, and a batch reconciliation path owns the money.
  49. Consistent hashingPlace keys and nodes on the same ring, and each key belongs to the next node clockwise. Adding a node then moves only the keys on its arc, not almost every key as with hash modulo N.

Traffic and messaging

  1. Layer 4 vs layer 7 load balancingWhat a layer 4 balancer can see compared with a layer 7 proxy, and why most production systems use both.
  2. Load balancing and proxiesHow a load balancer chooses which server gets each request, and how a badly set up balancer can cause the outage it was meant to prevent.
  3. Balancing algorithmsRound robin, least outstanding requests, power of two choices and consistent hashing, and where each one fails.
  4. Health checks and drainingHow a balancer detects a sick backend with active and passive checks, why deep health checks can empty a whole fleet, and how to remove an instance without dropping requests.
  5. Service discovery and service meshesHow clients learn which instances exist right now, the three discovery styles, client-side balancing, and what a service mesh costs.
  6. Global load balancingHow DNS and anycast steer a user to the nearest healthy region, and why DNS failover can take minutes.
  7. Speed of light and round tripsDistance sets a floor on latency that no server tuning can remove. A fresh HTTPS request pays several round trips before the first byte, which is why content has to move closer to users.
  8. CDN routing and cache hierarchyA CDN sends each user to a nearby point of presence, then uses tiers, an origin shield and request collapsing so that a new object reaches your origin about once, not once per PoP.
  9. CDNs and the edgeServe content from points of presence near users. A CDN cuts latency, absorbs attacks and cuts bandwidth costs, and its cache keys, headers and purge rules decide how well it works.
  10. Cache control and cache keysThe cache key says which requests count as the same object, and Cache-Control headers say how long to keep it. Get the key wrong and you waste the cache or leak private data.
  11. Purging and versioned URLsChange content in a CDN by giving it a new URL, purging exact URLs, or purging by tag. Pick the cheapest one that matches how many pages a change touches.
  12. CDN hit ratio and costCDN economics come down to egress price and hit ratio. Each point of hit ratio you gain cuts origin traffic, and a jump from 80% to 95% cuts origin load by a factor of four.
  13. Click counting off the hot pathWhy a redirect should never update a counter on the links table, and how an event queue and batch workers count clicks instead.
  14. Edge compute and securityCode that runs in CDN points of presence can reject bad requests and answer simple questions close to users. It's a poor fit for work that needs a distant database, and it also absorbs attacks.
  15. Polling and long pollingHow a client gets updates by asking, why short polling wastes almost every request, and how long polling holds each request open until there is news.
  16. SSE and WebSocketsServer-Sent Events push a one-way stream over plain HTTP with built-in resume, WebSockets give full-duplex frames, and WebTransport adds streams and datagrams over HTTP/3. Choose by direction and message rate.
  17. Real-time deliveryHow a server pushes updates to clients that did not ask, from polling to WebSockets, and the gateway, pub/sub and reconnect design that serves millions of open connections. Start here for the real-time map.
  18. Gateways and connection registryWhy real-time servers split into thin stateful gateways and stateless services, how a TTL registry finds the gateway holding a user, and how many connections one machine can hold.
  19. Fan-out with pub/subHow one message reaches thousands of connected users through a pub/sub backbone, why subscribing by interest beats routing to each user, and the write math for large channels.
  20. Connection lifecycleHow heartbeats find dead connections, how sequence numbers and resume make reconnects lossless, and how jittered backoff prevents a reconnection storm after a deploy.
  21. Synchronous vs asynchronous workThe difference between waiting for a task to finish and handing it off, and why that choice shapes latency, failures and user experience.
  22. Message queuesA durable buffer between the code that creates work and the code that does it. Start here for the map of queue concepts.
  23. Queues vs pub/sub topicsTwo ways to deliver a message. A queue gives each message to one worker. A topic gives a copy to every subscriber. Real systems use both.
  24. Delivery semanticsThe three promises a messaging system can make (at-most-once, at-least-once, exactly-once) and why the real choice is when the consumer says "done".
  25. Acks and visibility timeoutsHow a broker knows a message is still being worked on, what happens when a worker disappears, and how to set the timeout so work isn't duplicated.
  26. Message orderingWhy messages can arrive out of order, what it costs to guarantee order, and how to design consumers that don't need it.
  27. Retries and dead-letter queuesHow to retry a failing message with growing delays, when to stop, and where to put messages that will never succeed so the rest of the queue keeps moving.
  28. Backpressure and autoscalingA queue only stores work, it doesn't create capacity. Learn how to tell when the backlog is growing, how to scale workers on the right signal, and what to do when you can't add more.
  29. Choosing a queue productHow Amazon SQS, RabbitMQ, Google Cloud Pub/Sub and Redis Streams differ, and how to pick one based on the shape of your work rather than on brand names.
  30. Durable workflowsHow to run a multi-step process (charge, reserve, ship, notify) that must survive crashes, retries and undo steps, using a workflow engine instead of chained queues.
  31. The log as a data structureAn append-only, ordered sequence that many readers can consume at their own pace. Learn how a log differs from a queue, and how retention and compaction shape it.
  32. Kafka architecture and replicationHow Kafka splits a topic into replicated partitions, what makes a write durable, and how KRaft replaced ZooKeeper. Learn the pieces before you tune them.
  33. Event streaming and KafkaHow a durable log lets many consumers read the same events at their own pace, and how Kafka builds that log from partitions, replicas and consumer groups.
  34. Producers, keys and batchingA producer picks a partition with a key, batches records for speed, and sets acks for durability. Learn the settings that decide ordering, throughput and duplicates.
  35. Consumer groups, offsets and lagConsumers in a group split a topic's partitions, commit offsets to record progress, and fall behind when lag grows. Learn how commit timing sets delivery guarantees and how rebalances work.
  36. Exactly-once inside Kafka and its limitsKafka transactions give exactly-once processing from topic to topic, but the guarantee stops at Kafka's edge. Learn what it covers, and how sinks finish the job.
  37. Sizing partitions and avoiding hot keysPartition count sets your parallelism, and it is hard to change later. Learn to size it from throughput and storage, and to handle keys that carry too much traffic.

Distributed systems

  1. LinearizabilityThe strongest guarantee for one piece of replicated data. Once a write finishes, every later read sees it, and no reader can go back to an older value.
  2. CAP and PACELCCAP says that during a network split you must choose between answering every request and returning the latest value. PACELC adds the everyday trade-off that exists even when the network is fine.
  3. CAP, PACELC and consistency modelsWhat "consistent" means once data lives on several replicas, from linearizable to eventual, what each guarantee costs, and how to pick one per feature.
  4. Causal and session guaranteesWeaker promises than linearizability that still fix the bugs users notice. Causal consistency keeps related events in order, and session guarantees protect one client's view.
  5. Isolation vs consistencyIsolation and consistency both get called "consistent", but they answer different questions. Isolation is about overlapping transactions. Consistency is about whether replicas agree on the latest value.
  6. Choosing a consistency model per featureReal systems mix models. For each piece of data, ask what it costs when a reader acts on a stale value, then pick the cheapest model that avoids that cost.
  7. Partial failure and the unknown outcomeA remote call can fail in several ways that look identical from the caller's side. Learn why silence tells you so little, and why a retry can charge a customer twice.
  8. Network faults and gray failuresPackets are lost, delayed, duplicated and reordered, and partitions can be one-way or partial. Learn the faults to design for, including gray failures that pass health checks.
  9. System models and what you assumeA correctness proof depends on the assumptions behind it. Learn the three parts of a system model, network, node and timing, and the model real systems should design for.
  10. What goes wrong in distributed systemsPartial failure, unreliable networks, paused processes and timeouts, and the fencing tokens and failure detectors that let you build correct systems on top of them.
  11. Process pauses and clock jumpsA process can freeze for seconds and keep believing it is still the leader. Learn the causes of pauses and clock jumps, and why they break leases and locks.
  12. Timeouts and failure detectorsSince you can't tell dead from slow, you guess with timeouts. Learn how to pick them from measured latency, and how phi accrual detectors adapt to a jittery network.
  13. Fencing tokens and why locks are hardA lock can expire while its holder is paused. Learn how fencing tokens let the storage reject a stale holder, and why the Redlock debate matters.
  14. Metastable failures and retry stormsSome outages keep going after their trigger is gone, because the system's own reactions keep it overloaded. Learn the three parts of a metastable failure and how to break the loop.
  15. Clock drift and NTPEvery server's clock drifts, and NTP only corrects it within the limits of the network path. A clock is an estimate with an error bar, not an exact reading.
  16. Monotonic vs wall-clock timeWall-clock time tells people when something happened but can jump. Monotonic time only moves forward, so use it to measure how long things take.
  17. Time, clocks and orderingWhy wall clocks lie across machines, how skew silently loses writes, and how Lamport clocks, vector clocks, hybrid logical clocks and TrueTime order events you can't time directly.
  18. Last-write-wins and clock skewLast-write-wins keeps whichever write carries the higher timestamp. When clocks disagree, that is not the write that happened last, and the loss is silent.
  19. Lamport clocksA Lamport clock is a counter that moves forward on every event and jumps past any message it receives. It gives an order that respects cause and effect, without any physical clock.
  20. Vector clocksA vector clock keeps one counter per process. Comparing two vectors tells you exactly whether one event happened before another, or whether they were concurrent.
  21. Hybrid logical clocks and TrueTimeHybrid logical clocks combine physical and logical time, so timestamps stay near real time and respect cause and effect. TrueTime makes clock uncertainty explicit, then waits it out.
  22. Unique ID generation across shardsWays to create IDs that stay unique across many shards without a central counter, compared by size, time ordering and coordination. Snowflake packs a timestamp, a worker ID and a counter into 64 bits.
  23. Majority quorumsA decision counts only when more than half the nodes accept it. Because any two majorities overlap, two conflicting decisions can never both win. This is why consensus clusters have 3 or 5 nodes.
  24. Consensus and RaftHow a cluster agrees on one value or one log despite crashes and partitions, how Raft works, and where etcd, ZooKeeper and consensus-backed databases fit.
  25. Raft leader electionHow a Raft cluster picks one leader. Followers wait for heartbeats, randomized timeouts start an election, and voters refuse candidates whose logs are behind.
  26. Raft log replicationHow a leader writes a command to a majority before it commits, how followers are repaired to match the leader's log, and why committed entries survive leader changes.
  27. What consensus costsEvery committed write needs a round trip to a majority plus a disk flush. Across regions, distance dominates. A single group is one leader, so you scale by running many small groups.
  28. The dual-write problemWriting to two systems without a shared transaction leaves a window where one write has happened and the other has not. Learn the ways this goes wrong and how to avoid the window.
  29. Two-phase commitThe classic protocol for atomic commits across several databases. Learn how prepare and commit work, why a crashed coordinator blocks everyone, and how distributed SQL databases reduce that risk.
  30. Distributed transactionsKeeping data consistent when one business operation spans several databases or services, using two-phase commit, sagas, the transactional outbox and reservations, and redrawing boundaries first.
  31. Sagas and compensating actionsA saga runs a business operation as local transactions, each with an undo step. Learn how compensations work, how to order the steps, and why sagas are atomic eventually but not isolated.
  32. Orchestration versus choreography for sagasA saga can be driven by a central orchestrator, or by services reacting to each other's events. Learn the trade-offs, and why durable workflow engines make orchestration practical.
  33. The transactional outboxUpdate a database and publish an event without a dual write. Learn how the outbox commits the event with the change, how a relay publishes it, and how an inbox removes duplicates.
  34. Reservations and try-confirm-cancelFor scarce resources such as seats, rooms and stock, split each step into a hold and a confirmation. Learn how TCC uses expiring holds instead of database locks.
  35. Concurrent writes and last-write-winsA conflict happens when two replicas accept writes that neither has seen. Last-write-wins always picks one, but it silently drops the other write and trusts clocks that drift.
  36. Conflict resolution and CRDTsWhat to do when replicas accept concurrent writes. The options run from last-write-wins and siblings to CRDTs, operational transformation and the sync engines used by collaborative editors.
  37. CRDT countersA conflict-free replicated data type lets replicas update independently and still converge. A G-counter keeps one slot per replica and merges by taking the maximum, so no increment is lost or counted twice.
  38. OR-sets and add-winsSets are where naive merges break. An observed-remove set tags every add with a unique ID, so a remove only deletes what its replica has seen. A concurrent add survives.
  39. Text editing: OT vs sequence CRDTsWhen two people type at the same spot, position-based edits diverge. Operational transformation fixes positions on a central server, and sequence CRDTs give each character an ID so no server is needed.
  40. Design a collaborative editorA design exercise for a rich-text editor where many people type at once, see each other's cursors, work offline and keep history. Local CRDT updates, a sync server with an update log, and periodic snapshots.
  41. Check-then-act racesWhy reading a seat's status and then writing it back can sell the same seat twice, and how one conditional UPDATE turns the check and the claim into a single atomic step.
  42. Quorum reads and writesWith N replicas, a write succeeds after W of them acknowledge, and a read waits for R. When R + W is greater than N, every read overlaps every successful write in at least one replica.
  43. Sloppy quorums and hinted handoffWhen a home replica is down, Dynamo writes to a stand-in node and attaches a hint naming the intended owner. Writes keep working, and the hint is handed back when the owner returns.
  44. Merkle anti-entropy and read repairTwo ways replicas heal after a missed write. Read repair fixes keys when someone reads them. Merkle-tree anti-entropy finds and fixes cold keys by comparing hashes, sending only the ranges that differ.
  45. Dynamo vector clocksDynamo tags each version with a vector clock, a list of counters per writer. If one clock dominates another, the newer version wins. If neither dominates, the writes were concurrent, and both are kept as siblings.

Architecture patterns

  1. Window counting limitersFixed windows, sliding logs and sliding counters, and the boundary burst that makes the simplest one leak.
  2. Token bucketA bucket that holds tokens and refills over time, giving you two dials for a sustained rate and a burst size. The most common shape for API limits.
  3. Rate limitingWhy systems cap how much work a client can send, which algorithm counts, where the limit lives and what the client is told. Start here for the rate-limiting map.
  4. Distributed limiter on RedisHow many servers share one rate-limit count without races, using an atomic Redis Lua script, one clock, expiring keys and a deliberate failure mode.
  5. Local plus global limitsHow high-volume limiters avoid a Redis call per request by deciding locally and syncing every few hundred milliseconds, and what overshoot that costs, worked through a 1 million requests per second design.
  6. Limit responses and quotasWhat a rate-limited client should receive (429, Retry-After and budget headers), and why a quota is a different, durable kind of limit.
  7. Token limits for LLM APIsWhy LLM APIs limit tokens per minute as well as requests, and how a gateway reserves the worst-case output up front and refunds the unused part.
  8. Why duplicates happenRetries at every layer and at-least-once queues mean the same request can arrive twice. Learn where duplicates come from, and why the receiver must tolerate them.
  9. Naturally idempotent operationsSome operations are safe to repeat by design, such as absolute sets, upserts and conditional writes. Learn to spot them, and to redesign the rest before you add keys.
  10. Idempotency and retriesMaking operations safe to retry, with idempotency keys, deduplicating consumers and jittered backoff, so at-least-once delivery never means a double charge.
  11. Idempotency keysFor operations that can't be made idempotent by design, a client-chosen key lets the server recognize a retry and replay the first answer. Learn the protocol and its failure cases.
  12. Deduplicating in message consumersA consumer can see the same message twice. Learn three ways to make a duplicate harmless, and how the inbox and outbox patterns fit together.
  13. Retry policy and backoff with jitterIdempotency makes retries safe, and a retry policy makes them useful. Learn which errors to retry, how to space attempts with exponential backoff and full jitter, and when to stop.
  14. Retry budgets and retry amplificationRetries at several layers multiply load exactly when a system is weakest. Learn the multiplication, and how to cap it with one retry layer and a retry budget.
  15. Cascading failuresHow one slow dependency exhausts threads, then turns into a full outage through retries and overload, with the arithmetic that shows why.
  16. Timeouts and deadlinesA timeout caps how long one call waits, and a deadline caps the whole request. Set timeouts from latency data, and pass the remaining budget down every hop.
  17. Retry budgets and hedgingWhen a retry helps and when it multiplies load, how to space retries with jittered backoff and a budget, and how hedged requests cut tail latency for reads.
  18. Resilience patternsThe toolkit for keeping one failure from becoming an outage, from timeouts and retries to circuit breakers, bulkheads, load shedding and cells. Start here for the resilience map.
  19. Circuit breakersA circuit breaker watches a dependency's recent failures, trips to fail fast when the failure rate is high, and probes again after a cool-down.
  20. Bulkheads and isolationHow giving each dependency or workload its own bounded pool of resources stops one failing part from taking the whole service down.
  21. Load shedding and degradationHow to reject excess work early and by priority, so the work you accept still finishes on time, and how to serve a worse but usable experience instead of an error.
  22. Limiting blast radiusArchitecture that keeps a failure small by construction, using static stability, cells and shuffle sharding, with the arithmetic behind each.
  23. Monolith vs microservicesThe spectrum from monolith to modular monolith to microservices, what each one costs, and how Shopify and Segment reached opposite answers.
  24. Conway's law and team shapeWhy a system tends to copy the communication structure of the team that builds it, and how microservices mostly help by letting teams ship on their own schedules.
  25. Bounded contextsHow to find service and module boundaries by splitting along business meaning, and why splitting by technical layer almost always fails.
  26. Monoliths and microservicesWhen to split a system, how to draw service boundaries, and the operational cost that microservices add.
  27. Service communication: synchronous vs asynchronousHow synchronous calls and asynchronous events couple availability differently, how failures multiply down a call chain, and how a distributed monolith forms.
  28. Database per serviceWhy each service should own its data, what a shared database costs, and how to live with duplicated data, no cross-service joins and no cross-service transactions.
  29. The operational tax of microservicesWhat a team needs in place before it splits into services, including discovery, tracing, per-service pipelines, contract tests, versioning, resilience and service identity.
  30. Strangler fig migrationHow to move from a monolith to services one capability at a time, with a router in front, and why "monolith first" is the default advice.
  31. Events, commands and notificationsCommands ask one receiver to act, while events announce facts that anyone can react to. Learn the difference in coupling, and how thin and fat events trade consistency for independence.
  32. Change data captureCDC turns committed database changes into events by reading the database's own replication log, which avoids dual writes. Learn how it works, and why an outbox table usually sits in front of it.
  33. Event-driven architecture, CQRS and event sourcingBuilding systems around events, from change data capture to event sourcing and read models, and knowing when it simplifies a system versus creating a distributed mess.
  34. Event sourcingStore the events as the source of truth and derive the current state by replaying them. Learn the gains in audit and replay, and the costs of snapshots, schema changes and querying.
  35. CQRS and read modelsCommand Query Responsibility Segregation uses separate models for writing and reading. Learn how read models are built from events, the eventual consistency they bring, and how to hide that lag from users.
  36. Pitfalls of event-driven systemsEvent-driven systems fail in predictable ways, including hard debugging, ordering gaps, duplicates, event soup and personal data in immutable logs. Learn the fix for each before the first event ships.
  37. Worked example for an order system on an event logDesign an order system around a durable event log, with sizing, consumers and failure scenarios. Learn how one event stream feeds search, analytics and email without coupling them.
  38. Tenant isolation modelsThe spectrum from shared tables to a dedicated stack per tenant, what each model costs, and the enterprise requirements that push large customers toward more isolation.
  39. Multi-tenant SaaS architectureDesigning one system for many customers, covering isolation models, tenant scoping, noisy neighbors, tenant-aware sharding and per-tenant cost.
  40. Enforcing tenant scopingHow to make sure no query in a shared database returns another tenant's rows, using the tenant from login, automatic scoping, Postgres row-level security and leak tests.
  41. Noisy neighborsHow one tenant's heavy load can slow everyone else, and the controls that contain it, from per-tenant limits and fair queuing to dedicated pools and shuffle sharding.
  42. Moving tenants and shardingHow to shard a multi-tenant database by tenant, use logical shards and a directory, and move a large tenant online without downtime.
  43. Cost per tenantHow to meter each tenant's use of shared resources, allocate the cloud bill back to tenants, and find the customers who cost more than they pay.
  44. Why go multi-regionThe three legitimate reasons to run in several regions (latency, regional disaster tolerance and residency), the costs they bring, and why multi-zone deployment comes first.
  45. Active-passive, RPO and RTOHow a standby region takes over, what RPO and RTO mean, and how replication lag and human steps set the numbers.
  46. Multi-region architectureActive-passive, active-active and cell-based designs for latency, disaster tolerance and data residency, with the recovery, consistency and cost trade-offs each one carries.
  47. Active-active and home regionsHow every region can serve live traffic by giving each user or tenant a home region that owns its writes, plus the capacity math for surviving a region loss.
  48. Globally consistent databasesWhich data needs one consistent answer everywhere, how Spanner, CockroachDB, Aurora Global and DynamoDB pay for it, and why the cost lands on every write.
  49. Cells and regional isolationHow regions share as little as possible, and how cells cap the damage from a bad deploy or config push to a small slice of users.
  50. Failover testingWhy an untested failover plan is close to no plan, what the GitHub 2018 incident shows about failover going wrong, and the practices that keep failover working.
  51. Scaling reads and writesRead load is spread with copies, caches and CDNs. Write load is spread by splitting the data, queuing spikes and using stores built for fast appends. Each tool trades freshness or simplicity for capacity.
  52. Hot keys and celebritiesOne key or user can attract far more traffic than the rest. Sharding cannot spread a single key, so you split it, cache it, or change how updates fan out.
  53. Contention and holdsWhen many clients change one scarce unit, such as a seat or a coupon, use a conditional update with a time-limited hold, plus a waiting room. Locks are simple to explain, but they fail under a sudden spike.
  54. Multi-step consistencyWhen two systems must agree, such as inventory and payment, use a saga with compensating steps, an outbox for events, idempotency keys on every hop, and a reconciliation job. Two-phase commit everywhere is rarely the answer.
  55. Realtime updates and long-running workClients need fresh state without polling hard. Push it over a live channel. Work that outlasts one request runs as a job, with a status resource and a webhook when it finishes.

Case studies

  1. The system design interview frameworkA repeatable 45-minute structure, the signals interviewers score, and what changes between SDE2 and SDE3.
  2. Requirements and estimates in an interviewThe first eight minutes of a design interview. Ask the questions that change the design, state your assumptions, and compute only the numbers that justify a component.
  3. APIs and data models in an interviewDefine the interface and the entities in about ten minutes. Learn what a strong API signals, and how to pick a store for each entity from its access pattern.
  4. High-level design, deep dives and trade-offsDraw a design that works end to end, then pick two or three hard problems and solve each with options, a trade-off and a choice. Learn how to narrate the design and close the interview.
  5. What interviewers score and how levels differInterviewers score four dimensions and decide your level mostly in the deep dives. Learn the signals that separate levels, and the mistakes that sink strong candidates.
  6. Case study: URL shortenerThe classic warm-up: ID generation, read-heavy caching, 301 vs 302 redirects, and analytics without slowing the hot path.
  7. Short code generationThree ways to mint unique short codes (hash the URL, count and encode, or hand out random codes) compared by collisions, guessability and coordination.
  8. News feed requirements and estimatesWhat a home feed must do, the traffic and storage numbers that shape it, and the API and data model that follow from those numbers.
  9. Fan-out on write and fan-out on readFan-out turns one event into many updates. Push does the work when a post is written, pull does it when a feed is read, and the cost moves rather than disappears.
  10. Case study: news feedHow a home feed copes with fan-out, celebrities, ranking and deletes at hundreds of millions of users, and where the design gets hard.
  11. Timeline storageWhy a timeline is a sorted set of post IDs in Redis, how a feed page is hydrated from it, and why a lost timeline can be rebuilt.
  12. Hybrid fan-out and celebritiesPush for ordinary authors, pull for celebrities, and merge the two on read. This is how a feed handles a post from an account with 100 million followers without writing 100 million timeline entries.
  13. Feed rankingHow a ranked feed narrows candidates, scores them and diversifies the result, plus the seen-state filter that stops repeats.
  14. Timeline consistency, deletes and blocksHow a feed stays acceptable while fan-out is asynchronous, covering read-your-writes, tombstones for deletes, filters for blocks and mutes, and backfill for new follows.
  15. Chat requirements and scaleBefore you design a chat system, pin down what it must do and how big it is. The numbers set the shape, and one question about encryption and search changes almost everything.
  16. Client message IDsNetworks retry, so a chat message can arrive twice. The client assigns an ID before sending, the server stores it on the message, and duplicates are dropped by that ID.
  17. Presence and typingOnline status and typing indicators are best-effort. They're lost easily and recomputed often, so the system can drop them under load without losing a single message.
  18. Case study: chat systemDesign a WhatsApp or Slack style messaging system. Connection gateways, per-conversation ordering, wide-column storage and large-channel fan-out are the parts that matter.
  19. Chat connection layerWebSocket gateways hold each user's open connection, and a session registry maps users to gateways. Keeping history out of the gateways lets any gateway serve a user after a reconnect.
  20. Chat message orderingChat orders messages per conversation with a sequence number issued per channel, not with wall-clock time. A device that sees a gap backfills it from history before it applies the newer message.
  21. Chat storageChat history is append-heavy and read as recent messages in one channel. Wide-column stores, partitioned by channel and time bucket, fit that pattern. Attachments and search live outside the send path.
  22. Large channels and receiptsSmall groups can loop over every member and push to each one. Large channels need a topic that gateways subscribe to, and read receipts should be stored as watermarks, not one write per member per message.
  23. The notification pipelineThe stages a notification passes through, from the ingress API to a provider, with per-channel queues, a channel adapter layer and a durable in-app inbox.
  24. Preferences, quiet hours and digestsHow per-user preferences decide whether, where and when a notification is sent, including quiet hours, digests, channel cost and the compliance rules for opt-outs.
  25. Idempotency for notificationsHow an idempotency key lets a notification system recognize a repeat request, and why the realistic goal is at-most-one send per key rather than exactly-once delivery.
  26. Case study: notification systemDesigning push, email and SMS delivery at scale, covering the pipeline, preference checks, idempotency, provider rate limits, retries and priority lanes.
  27. Rate limits and provider quotasHow token buckets keep a notification system under each provider's limit, where limits belong in the design, and what to do when a bucket is empty.
  28. Retries and provider failuresHow to treat each kind of provider failure differently, with backoff and jitter for transient errors, token cleanup for dead devices, failover, and dead-letter queues for what keeps failing.
  29. Priority lanes and broadcastsHow separate lanes keep security messages fast during a marketing blast, how a broadcast expander fans out to millions of users in chunks, and the kill switch that protects critical paths.
  30. Templates and localizationHow templates turn a template ID, a locale and data into the text a user reads, with typed variables, locale fallbacks, version pins, escaping and failure into the dead-letter queue.
  31. Prefix top-k trieA trie stores strings by shared prefixes, and each node can keep its best completions. A lookup walks the prefix and reads one list, no matter how many queries exist.
  32. Case study: search autocompleteTypeahead under 100 ms: tries with cached top-k, offline query-log aggregation, prefix sharding and client debouncing.
  33. Offline query aggregationHow autocomplete scores come from batch counts of query logs, with a streaming overlay for trending phrases and versioned trie snapshots that can roll back.
  34. Keystroke latency budgetThe autocomplete target is about 100 ms from keystroke to pixels. Debouncing, edge caching and client caches spend that budget, and the origin trie alone cannot meet it on a mobile network.
  35. Geospatial indexesHow geohashes, quadtrees and H3 hexagons turn "who is near this point?" into a small set of cell lookups, and the boundary trap that catches naive geohash searches.
  36. Case study: ride-sharingUber-style dispatch: streaming driver locations into a geo index, matching under contention, trip state and geographic sharding. Start here for the ride-sharing map.
  37. Driver location ingestionHow a stream of driver location updates, about 250,000 per second for a large region, reaches an in-memory geo index, and why the matchable set holds only available drivers.
  38. Exclusive driver assignmentHow two riders competing for the same driver are settled with an atomic claim that expires, and how offers and retries avoid double bookings.
  39. Trip state machineHow a trip moves through explicit states from request to payment, why only legal moves are allowed, and how a payment on completion stays idempotent.
  40. Geo sharding and hot spotsHow to partition ride-sharing data and capacity by geography, contain failures to one city, and handle predictable surges at airports and stadiums, including how surge pricing damps demand.
  41. Case study: ticket bookingTicketmaster-style inventory under extreme contention, with holds that expire, a virtual waiting room and atomic seat commits so one seat is never sold twice.
  42. Timed holds and lazy expiryA hold keeps seats for about ten minutes while a buyer pays. Storing the expiry time on each row lets seats free themselves at claim time, without a cleanup job.
  43. Virtual waiting roomA waiting room admits buyers at a rate the booking system can handle. Signed, short-lived tokens keep bots and forged requests out of the hold path.
  44. Authorize, commit, captureThe order of checkout steps decides whether a buyer can be charged for a seat they lost. Authorize the card, commit the seat, then capture the money, and undo each step on failure.
  45. Payment intents and statesHow a PSP payment intent moves through its states, why every confirm can end in an unknown outcome, and why your own status column should be the one you trust.
  46. Idempotency at every hopHow idempotency keys protect a payment at each boundary, from the client to your API, from your service to the PSP and from the PSP's webhooks back to you, and why the key row must share the payment's transaction.
  47. Case study: payment systemMoving money correctly, with payment intents, idempotency at every hop, a double-entry ledger, webhooks and reconciliation when the network lies. Start here for the payments map.
  48. Double-entry ledgerHow a ledger records every money movement as balanced, immutable journals in integer cents, including refunds and payouts, and why balances should be derived from journals.
  49. Webhooks and outboxHow to receive PSP webhooks safely, with signature checks, fast acks and deduplication, and how the transactional outbox makes calls to the PSP survive a crash.
  50. ReconciliationHow to compare your ledger with the PSP's settlement records on a schedule, match each payment, flag the gaps and treat unexplained differences as bugs.
  51. Video segments and manifestsA video is stored as many short, immutable segments, listed in a manifest. The player downloads the small manifest first, then fetches segments in order from the nearest cache.
  52. Case study: video streamingYouTube and Netflix-style delivery, with resumable uploads, parallel transcoding into a bitrate ladder, HLS and DASH adaptive streaming, and CDN economics.
  53. Transcoding as a DAGTurn one uploaded master file into many renditions by splitting it into chunks and encoding each chunk in parallel, as a directed acyclic graph of small, retryable tasks.
  54. Adaptive bitrate streamingThe player measures its bandwidth and buffer after each segment, then picks the quality of the next segment, so playback keeps going on a slow network.
  55. Case study: distributed key-value storeBuild a Dynamo-style store from first principles. Consistent hashing places the data, quorums set the consistency, vector clocks detect conflicts, and hints and Merkle trees repair failures.
  56. The URL frontierThe URL frontier is the crawler's queue of pages still to fetch. Front queues decide what matters most, and the politeness rules decide when each host can be fetched.
  57. Case study: web crawlerCrawl billions of pages politely, with a Mercator-style frontier, DNS and robots caches, Bloom and SimHash deduplication, and defenses against spider traps.
  58. Politeness and back queuesOne queue per host, plus a heap of next-allowed fetch times, lets many fetcher threads stay busy while no site is contacted faster than its politeness delay.
  59. Deduplicating URLs and contentCrawlers deduplicate twice. A Bloom filter checks URLs already seen, and fingerprints (exact hashes or SimHash) check whether the page content is already stored.
  60. Spider traps and recrawl schedulingSpider traps are endless URL patterns that waste crawl budget. Per-host caps, pattern detection and change-based recrawl schedules keep the budget on pages that matter.
  61. TTFT and token streamingThe two numbers that define a chat response, time to first token and the speed of later tokens, and how streaming, prefill, decode and proxies shape each one.
  62. LLM capacity mathHow to turn daily users and tokens into requests per second, tokens per second and GPU counts, with an example that shows how cost and fleet size follow from the same numbers.
  63. Case study: LLM chat productDesigning a ChatGPT-style product, covering the request path, token streaming, conversation context, GPU capacity math, token-based rate limits, tools and untrusted input, and what to do when capacity runs short.
  64. Conversation context assemblyHow a chat product stores conversations as turns and references, then builds a bounded prompt on every request from the system prompt, recent turns, a summary, retrieved passages and file references.
  65. LLM rate limits and overloadHow to limit LLM traffic by tokens rather than requests, admit work by tier, and degrade in steps when GPUs run short, instead of failing every request at once.
  66. Tools and untrusted inputHow chat products let models call tools and read uploaded files safely, with bounded tool loops, timeouts, allow-listed actions and the rule that everything the model reads is untrusted.
  67. Streaming contract and cancellationThe contract between server and client for a streamed answer, covering event types, resumable streams, cancellation that stops GPU work, and partial answers saved on disconnect.
  68. Recurring deep-dive patternsA cheat sheet that maps hard problems, such as hot keys, contention, large media, realtime, long work and multi-step consistency, to proven solutions across the case studies.

Infrastructure, cloud and SRE

Running software in production: Linux, cloud, Kubernetes, delivery, observability, reliability, security and cost.

Linux and networking

  1. Shell pipes and streamsHow programs talk through stdin, stdout and stderr, and how pipes chain small tools into one answer to a production question.
  2. Processes and signalsWhat a process is, how to read its state, and how signals and exit codes tell you why it stopped. Exit 137 and 143 come up constantly.
  3. Permissions and capabilitiesHow Linux decides who can read, write or enter a file, why "permission denied" often comes from a parent directory, and how to grant one power instead of root.
  4. Disk space and mounts"No space left on device" has three different causes, only one of which df -h shows. Also how disks attach to folders with mounts.
  5. Linux for engineersThe parts of Linux you touch in production, from shell pipelines and systemd services to permissions, disks, network debugging commands and safe remote access.
  6. Safe bash scriptsBash fails quietly by default. A few lines at the top of every script turn unset variables and hidden failures into loud errors.
  7. Graceful shutdownHow orchestrators stop a process with SIGTERM, then SIGKILL after a grace period, and how a service drains in-flight work inside that window.
  8. systemd and journaldHow systemd runs a service as a unit with restarts and hardening, and how the journal captures its logs so you can query them by service, time and priority.
  9. Debugging connectivityWhen A cannot reach B, check each layer from the bottom up and write down what each one said. Timeouts and refusals point to different causes.
  10. The USE methodA checklist for finding a bottleneck: for every resource, check utilization, saturation and errors before you guess at the application.
  11. CPU load and the run queueWhat load average really counts, how the run queue shows CPU saturation, and why steal time in the cloud is a problem you cannot fix inside the guest.
  12. Memory, page cache and OOMWhy "free" memory looking low is normal, how to read available memory, and why swap and OOM kills are the two signals that memory is actually short.
  13. Linux performance analysisThe USE method and the 60-second checklist for finding why a box is slow, covering CPU, memory, disk, network and the container limits that look like application bugs.
  14. The 60-second checklistNine standard commands that give a fast USE pass over a Linux host in about a minute, and how to read what each one shows.
  15. Disk and network saturationHow to read disk utilization, await and queue depth, and how network retransmits, drops and listen overflows show a path that is overloaded.
  16. Profiling with perf and eBPFOnce a resource is confirmed as the bottleneck, sampling profilers and eBPF tools show which functions burn the CPU or wait on I/O, without rewriting the app.
  17. CIDR and VPC basicsA VPC is an isolated address space. CIDR notation says how big each part of it is. Plan the ranges before you create anything, because overlaps are hard to undo.
  18. Cloud networking and VPCsVPCs, subnets, routing, NAT, security groups and private connectivity in AWS and GCP, plus a three-AZ reference architecture and how to debug when A cannot reach B.
  19. Routes, internet gateways and NATA route table picks the next hop for each packet. Public subnets reach the internet through a gateway. Private subnets get outbound access through NAT, and endpoints skip NAT for cloud APIs.
  20. Security groups and NACLsSecurity groups are stateful firewalls on each instance and are the main control. Network ACLs are stateless filters on each subnet. Know which one you are editing, and reference groups instead of ranges.
  21. Connecting VPCs and private endpointsVPC peering links two networks, a transit hub links many, and PrivateLink publishes one service. Endpoints keep traffic to cloud APIs off NAT and the internet.
  22. Load balancers, DNS and the edge in the cloudConfiguring the traffic layer, from DNS and cloud load balancers through TLS, WAF and CDN to a regional traffic shift. Start here for the map of edge pages.
  23. Cloud load balancersWhat a cloud load balancer does, how ALB, NLB and Gateway LB differ, why health checks can take the whole fleet down, and where Ingress fits inside the cluster.
  24. DNS routing and TTLHow DNS routing policies steer traffic between endpoints, and why a TTL sets the real speed of a failover. Covers weighted, latency and failover routing, and the cache trap.
  25. TLS, WAF and CDN at the edgeWhere TLS ends, how certificates are renewed, and how a WAF, DDoS protection and a CDN sit at the outermost edge to protect origin servers and cut latency.
  26. Shifting traffic between regionsA step-by-step runbook for moving users from one region to another with weighted DNS, health checks and a fast rollback. Walks through the numbers and a real rollback decision.

Containers and orchestration

  1. Containers vs virtual machinesA virtual machine boots its own kernel. A container shares the host kernel. That one difference explains the speed, the size and the security trade-off.
  2. Namespaces and cgroupsA container is a Linux process with two kernel features applied. Namespaces control what it can see, and cgroups control what it can use.
  3. Containers from the insideNamespaces, cgroups and layered images - what a container really is, how images are built and shipped, and how to write small, fast, secure Dockerfiles.
  4. Image layers and digestsAn image is a stack of read-only layers named by their content hash. Learn why deleting a file does not shrink an image, and why production should pin digests, not tags.
  5. Writing good DockerfilesOrder Dockerfile steps so the cache works for you, build in stages so compilers stay out of the final image, and run as a non-root user.
  6. Container securityA container is a convenience boundary, not a strong security boundary. Defend it in layers, with minimal images, non-root users, read-only filesystems, dropped capabilities and no secrets in layers.
  7. Shrinking an imageA worked example that takes a 1.2 GB Python image under 150 MB. Measure each layer first, then fix the largest costs one at a time.
  8. The control plane and the reconciliation loopThe parts that run a Kubernetes cluster, and the loop that compares what you asked for with what is running and closes the gap. Includes the path a deployment takes to become running pods.
  9. Kubernetes core conceptsThe control loop at the heart of Kubernetes, the objects you use every day, and how a request reaches a pod. Start here for the map of Kubernetes pages.
  10. Pods and workload objectsA pod is the smallest unit you deploy. Workload objects wrap pods with rules for how many run and what happens on failure. Use the one whose rule matches your app.
  11. Services and DNS inside the clusterHow a Kubernetes Service gives a set of pods one stable name and virtual IP, how DNS and readiness route traffic, and what a request passes through on its way to a pod.
  12. Config, storage and labels in KubernetesWhere pods get their settings, secrets, disks and identity, and how labels and namespaces organize everything. Includes the storage rules that decide where a pod can run.
  13. Container throttling and cgroupsContainers share the host's kernel but live under cgroup limits. CPU throttling and memory limits can make a healthy node look like a slow app, and the fix starts with the numbers.
  14. Debugging CrashLoopBackOffA step-by-step order for finding out why a container keeps crashing, including exit codes, previous logs and bad probes. Use it before you guess.
  15. Resource requests, limits and QoSRequests decide where a pod fits. Limits decide what happens when it uses too much. CPU over its limit is throttled, while memory over its limit is killed.
  16. Health probesStartup, readiness and liveness probes decide when a container gets traffic and when it gets restarted. Liveness should be shallow, because a deep check can turn one dependency blip into a full restart storm.
  17. Kubernetes in productionRequests and limits, probes, autoscaling, disruption budgets, packaging and the mistakes that cause outages.
  18. Autoscaling pods and nodesThe horizontal pod autoscaler changes replica counts, the cluster autoscaler or Karpenter adds nodes, and KEDA scales on events. Choose the metric that matches how the service actually waits.
  19. Disruption budgets and spreading replicasA PodDisruptionBudget limits how many pods a drain or upgrade can take down at once. Topology spread and anti-affinity keep replicas on different nodes and zones.
  20. Rollouts and graceful terminationHow a rolling update swaps pods safely, and the order of events when a pod is deleted. Traffic has to stop before the process does, and requests longer than the grace period get cut off.
  21. Packaging, GitOps and operatorsKustomize, Helm and GitOps decide how Kubernetes manifests are packaged, reviewed and applied. Pin images by digest, and treat operators as code you still need to understand.
  22. Three kinds of serverlessFunctions, containers and managed platforms all count as serverless, but they hand you different things to ship and hide different amounts of the server.
  23. Serverless and managed runtimesLambda, Cloud Run and friends - what you give up, what you gain and the cost crossover versus always-on containers.
  24. Cold startsA cold start is the delay before a new serverless environment can serve its first request. Learn its three parts and the mitigations that actually cut it.
  25. GPUs on KubernetesHow a GPU becomes a schedulable resource in Kubernetes, through the GPU Operator, tainted node pools and nvidia.com/gpu requests, and how MIG and time-slicing pack more work onto each card.
  26. Serverless event sources and retriesFunctions run when events arrive from queues, storage, schedules and webhooks. Those events are delivered at least once, so every handler must tolerate duplicates.
  27. Concurrency and database connections in serverlessServerless scales by adding environments, and each one can open a database connection. Learn how that arithmetic exhausts Postgres, and what to do about it.
  28. Serverless cost crossoverServerless is cheap when traffic is spiky and expensive when it is steady and high. Compare the request and GB-second math with an always-on container before you choose.

Cloud and infrastructure as code

  1. Regions and availability zonesA region is a set of isolated data centres. An availability zone is one building block inside it. Design every tier to survive losing one zone, and size the spare capacity for it.
  2. Cloud primitives on AWS and GCPRegions, zones, compute, storage, managed databases and messaging, how the services map between AWS and GCP, and how to decide what to run yourself.
  3. VMs, spot and autoscalingPick a VM family from the bottleneck, buy spot capacity for interruptible work, and let an autoscaling group keep a fleet at a target load.
  4. Storage typesBlock, object and file storage differ in who can attach them, how they are priced and how fast they respond. Picking the wrong one is a common and expensive mistake.
  5. Managed databases and messagingThe managed database, cache, queue and streaming services on AWS and GCP, what each one is good at, and the limits that decide your design.
  6. Managed vs self-hostedFor each component, weigh the provider's managed service against running open-source software yourself, using operational burden, total cost and lock-in.
  7. Why infrastructure as codeInfrastructure as code keeps cloud resources in reviewed files. Learn the difference between declarative and imperative setup, and what drift is.
  8. Infrastructure as code with TerraformDeclaring infrastructure in code — providers, state, plans, modules, environments and the habits that keep apply safe.
  9. Terraform plan and applyThe Terraform loop. Write resources, read the plan, then apply it. Learn the building blocks and the plan symbols that flag replacements.
  10. Terraform state and lockingThe state file maps your code to real resources. Keep it in a remote backend with locking, and treat import, move and remove as production surgery.
  11. Terraform modules and environmentsModules package a standard into a reusable folder. Separate directories, each with its own state, keep dev and prod apart. Workspaces are useful but easy to misuse.
  12. Safe Terraform changesRun plans in CI, check policy before apply, and guard stateful resources in the config. Force-new changes need extra care because they can replace data.
  13. Twelve-factor configKeep config in the environment, not the image, so one tested artifact runs in every environment. Separate build, release and run, and promote the same image digest.
  14. Environments and parityMost teams need only a few environments. Each long-lived one adds cost and drift. Parity, meaning the same shape, is what makes a staging test count as evidence.
  15. Configuration, environments and feature flagsTwelve-factor config, environment parity, secrets injection, feature flags and the blast radius of a bad config push.
  16. Config layering and validationConfig comes from layers, and the most specific one wins. Load it once at startup into a typed structure, and refuse to boot when a required value is missing or wrong.
  17. Feature flagsA feature flag picks a code path at runtime. It lets you deploy code dark, ramp it safely and separate deploy from release. Keep flags short-lived, because stale flags are production hazards.
  18. Dynamic config and blast radiusRemote config and flags can change every process in minutes. That speed is useful for kill switches, and it is why config pushes cause as many outages as bad deploys. Roll them out in stages, validate them and keep a rollback.

Delivery

  1. Pipeline stages and cachingOrder CI checks from cheap to expensive, run independent jobs in parallel, and cache what has not changed. These three levers decide whether a pipeline takes 5 minutes or 40.
  2. Continuous integration and the software supply chainFast, trustworthy pipelines - build, test, artifact, scan and sign - plus DORA metrics and supply-chain controls that keep trunk green.
  3. Monorepos and affected buildsIn a monorepo, run only the packages a change touches and the packages that depend on them. A dependency graph and a remote cache make that cheap.
  4. Flaky testsA flaky test passes and fails with no code change. Detect it, quarantine it with an owner, and fix the root cause. Never silently retry it into green.
  5. Supply-chain securityThe pipeline is part of the software supply chain. Pin dependencies, generate SBOMs, sign artifacts, and use OIDC instead of stored cloud keys, so each hop can be checked.
  6. DORA metricsFour numbers that describe how well a delivery system works - deploy frequency, lead time, change failure rate and time to restore. They diagnose the system, not the people in it.
  7. Deploy vs releaseA deploy puts new bits on servers. A release changes what users experience. Keeping them separate lets you ship often and roll back the right thing.
  8. Deployment strategies and safe releasesRolling, blue-green and canary deploys, automated rollback, and zero-downtime schema changes - ship often without betting the fleet on every commit.
  9. Rollout strategiesRolling, blue-green, canary and shadow deploys replace a running version in different ways. Each trades speed, cost and blast radius differently.
  10. Automated canary analysisA canary needs gates, not a person watching a graph. Compare the canary with the baseline on agreed metrics over a bake period, then promote it or roll it back.
  11. Rollback and roll-forwardRollback restores the previous artifact or traffic shape. Roll-forward ships a fix on top of the broken release. Both are valid, and the choice depends on how fast you can restore service and whether the data has changed.
  12. Zero-downtime schema changesUse expand and contract to change a database while old and new app versions run together. Add compatible changes first, switch the code, backfill, and remove the old shape last.
  13. Cognitive load and team typesWhy product teams slow down when they must master cloud, CI and security themselves, and the Team Topologies vocabulary for splitting that work across team types.
  14. Golden pathsA golden path is an opinionated, maintained way to create, ship and observe a service. It should be the easy default, not a prison, and the test is whether a new hire can reach production in a day.
  15. Platform engineeringInternal developer platforms, golden paths and treating infrastructure as a product. Cut cognitive load without building a second Kubernetes.
  16. Catalogs and portalsA developer portal is the interface on top of the platform, and a service catalog answers who owns a service, where its code lives, and how to reach them at 3 a.m. Start with ownership, not with the portal's chrome.
  17. Platform as a productTreat internal engineers as customers, measure adoption and time to a safe production deploy, and choose a thin platform over a thick one until a clear need forces a thicker abstraction.
  18. Build vs buy for platformsMost platform parts should be bought or taken from open source. The homegrown PaaS trap is a custom YAML dialect that only three people understand, and that nobody funds to upgrade.
  19. Guardrails and scorecardsScorecards make drift visible. Guardrails make the unsafe path hard for the baselines that prevent company-scale incidents. Mandates should be rare and come with migration help.

Observability and reliability

  1. Structured logsA structured log is one machine-readable event with named fields. The few fields that always pay off are the request ID, trace IDs, a stable route, and a level.
  2. Monitoring versus observabilityMonitoring checks the questions you wrote down in advance. Observability lets you ask new questions of a running system, by slicing high-cardinality events after the fact.
  3. Wide eventsOne rich event per request, with dozens of fields, lets you find the cohort that is broken when every average looks healthy. You can start with a single structured log line.
  4. Observability: logs and tracesStructured logs, distributed tracing and OpenTelemetry, so you can ask new questions of a running system instead of guessing which dashboard to open.
  5. Logs, cost and privacyLogs are often the biggest observability bill, and they can leak secrets. Sample routine traffic, keep every error, never log secrets or PII, and do the volume math before you turn on debug logging.
  6. Distributed tracingA trace is the full path of one request across services, built from timed spans. Trace context has to be passed on at every hop, or the waterfall breaks into pieces.
  7. Trace samplingYou rarely keep every trace at scale. Head sampling decides at the start and is simple. Tail sampling decides at the end, so it can keep the errors and the slow requests.
  8. OpenTelemetryA vendor-neutral standard for producing traces, metrics and logs. Instrument once in the app, send to a collector, and choose the backend later without rewriting code.
  9. Metric types and rateThe four metric shapes (counter, gauge, histogram, summary) and why counters are always turned into a per-second rate before anyone reads or alerts on them.
  10. Labels and cardinalityEvery unique label combination in Prometheus is a separate time series. Unbounded labels such as user IDs can crash the metrics store, so keep those values in logs and traces.
  11. Metrics, dashboards and alertingGolden signals, RED and USE, Prometheus histograms and alerts that page you only when users are hurting.
  12. Histograms and fleet percentilesWhy you cannot average p99 latency across pods, and how to aggregate histogram buckets first and then take the quantile. Summaries cannot be merged, so they are the wrong tool for fleet SLOs.
  13. RED, USE and dashboardsThree frameworks for choosing what to graph, and a dashboard layout that answers "is this service okay?" in one screen. RED on top, USE underneath.
  14. Alerting on symptomsPage a human only for user pain, such as an error ratio or a burn rate, and send everything else to a ticket. Every page needs a runbook and a first step.
  15. Alert fatigue and cleanupWhen pages are frequent or useless, responders stop trusting them. Measure the alerting system itself, and run a regular cleanup loop that fixes, widens or deletes noisy rules.
  16. SLIs, SLOs and SLAsMeasure reliability as a ratio of good events to total events. Learn the three terms, and how to pick measures from what users notice.
  17. SLIs, SLOs and error budgetsDefine reliability in numbers users care about, then use error budgets and burn-rate alerts to balance shipping speed against stability.
  18. Error budgetsThe error budget is the failure your SLO allows. Turn it into numbers, and agree on a policy for what happens when it runs low.
  19. Burn-rate alertsA burn rate compares your error rate with the pace that would use the budget exactly over the window. Page on fast burns over two windows, to catch problems early without noise.
  20. Latency SLOs and histogramsA latency SLO counts requests that finish under a threshold. Build it from histogram buckets with an edge at the threshold, and never from averaged percentiles.
  21. Rolling out SLOs on a team with noneStart with one measured journey, let the data set the first target, then add alerts and a policy. A strict target that the team ignores is worse than none.
  22. Incident responseRun an incident with clear roles, severity and communication. Mitigate first, debug second, then hand off to the postmortem. Start here for the map of incident pages.
  23. Incident severity and rolesHow to rate an incident's severity so everyone shares the same urgency, and how to split the work into commander, operations, communications and scribe roles so one person is not doing everything.
  24. The first fifteen minutesThe order of actions in the first fifteen minutes of an incident, with a worked timeline for a database connection pool failure that ends in a rollback.
  25. Mitigate before root causeDuring an incident, restore service first using reversible, practiced levers, and find the root cause once users are no longer failing. Covers the mitigation menu, hypothesis timeboxing and risky-change sign-off.
  26. Incident communication and handoffsHow often to update during an incident, what to say to each audience, the status words to use, and how to hand off or close an incident cleanly.
  27. Blameless cultureBlameless postmortems ask which conditions made a failure possible, not who to blame. Accountability for fixes stays in place.
  28. Blameless postmortemsLearning from failure with timelines, contributing factors and action items that actually get done — without turning people into the root cause.
  29. Writing the summary and timelineStart with user impact, then build the timeline from machine data. Record the silent window between failure and detection, because it is often the most important line.
  30. Contributing factors, not a single root causeComplex failures come from several causes that combine. List factors by category, and do not stop at the first person, because that fixes nothing.
  31. Action items that get finishedGood action items are specific, owned, prioritized and dated. Tracking them is what turns a postmortem into an actual fix.
  32. Sharing postmortem learningsA postmortem that stays in one team's folder teaches one team. Build an incident library, run short reviews, and read public write-ups for structure.
  33. Rotation designHow to size an on-call rotation so people can sustain it, with a primary, a secondary, written handoffs, and a clear definition of what coverage means.
  34. Paging policyA page is an interruption that costs sleep, so reserve it for user pain, set an acknowledgment target, and route everything else to a ticket. Google's target is at most two incidents per 12-hour shift.
  35. Designing on-callSustainable rotations, executable runbooks, toil budgets and a paging policy that keeps humans healthy while production stays covered.
  36. Executable runbooksA runbook is a checklist a tired person can follow at 3 a.m. Each page-level alert needs one, with the user impact, quick checks, mitigations in order, and when to escalate.
  37. Toil and its budgetToil is manual, repetitive work that scales with the service and leaves nothing behind. Google's guidance is to keep it under about half of an SRE's time, and to automate the top offenders first.
  38. On-call health and fatigueOn-call costs sleep and focus. Healthy programs pay for it, cap consecutive nights, allow handoffs when sick, and review the load every month. Overload is a reliability defect.
  39. Ownership models for on-callDeveloper-owned on-call, dedicated SRE and hybrids each suit different systems. The failed middle is when SRE takes the pages for product bugs that product teams could fix.
  40. Demand forecastingTwo things drive demand, steady growth and discrete events such as launches. Plan against the peak you will actually see, and turn every marketing event into a load estimate.
  41. Utilization and the kneeQueues form when work arrives faster than it is served. Latency stays flat for a long time, then climbs steeply as utilization nears 100%. That bend is the knee, and it is why planning targets sit near 70 to 80%.
  42. Capacity planning and load testingForecasting demand, finding the latency knee with load tests and keeping headroom for peaks, deploys and AZ loss.
  43. Little's law and pool sizingLittle's law says in-flight work equals arrival rate times time in system. Use it to size thread and connection pools, and to see why a database, not the app, is often the real ceiling.
  44. Load testing for the kneeA load test finds the offered load where latency turns up sharply or throughput stops growing. Use open arrival rates, realistic traffic and soak runs, and read the generator's own metrics.
  45. Headroom for failureCapacity must cover a zone failing during peak, rolling deploys, slow autoscaling and retry storms. Size for the survivors, and keep them under the knee.
  46. Worked example - a 5x launchPlan a checkout API for a 5x traffic launch in five steps - app compute, failure headroom, the database, load-test proof and launch controls. Each step is arithmetic you can check.
  47. Disaster recovery and chaos engineeringRPO and RTO, backups that restore, game days and region drills, and deliberate failure injection, so confidence comes from measured recovery. Start here for the map of disaster recovery pages.
  48. RPO and RTOThe two numbers that shape every recovery plan. RPO is how much data you can lose, RTO is how long you can be down, and each one maps to a recovery tier with a price.
  49. Backups that restoreWhat makes a backup real, why restores must be timed, how point-in-time recovery works, and the GitLab 2017 outage that shows what untested backups cost.
  50. Game days and region drillsHow to rehearse a failover or region evacuation with the real team and runbook, write a testable hypothesis, measure the actual RTO and RPO, and include degraded dependencies in the plan.
  51. Chaos engineeringHow to inject controlled failures to test steady state, following the hypothesis, inject, measure, fix and automate loop, with blast-radius limits and abort criteria.

Security, cost and GPUs

  1. IAM and least privilegeHow cloud identity works, why least privilege is a habit rather than a checkbox, and where the org-level guardrails sit. Start here for the map of IAM pages.
  2. How IAM decides allow or denyThe order cloud IAM uses to decide a request. Explicit deny always wins, identity and resource policies both count, and no allow means no access.
  3. Least privilege in practiceGrant the narrowest permissions a job needs, then tighten them with real usage data. Covers the practical loop, a worked example and conditions that scale.
  4. Workload identity without long-lived keysHow running workloads prove who they are and receive short-lived cloud credentials, with no static key to leak. Covers IRSA, GCP Workload Identity and OIDC for CI.
  5. Org guardrails and human accessHow people log in through SSO with short sessions, how break-glass access works, and how organization policies stop one bad admin policy from becoming a breach.
  6. What a secret is and its lifecycleA secret is any credential that grants access. Each one moves through create, distribute, use, rotate, revoke and audit, and leaks happen when that path breaks.
  7. Secrets management and encryptionKeeping credentials out of code and data safe at rest and in transit with KMS, Vault and envelope encryption — plus how to catch leaks before they become incidents.
  8. Where secrets liveCompare cloud secret managers with Vault, and see why a Kubernetes Secret is not a vault by default.
  9. Envelope encryption with KMSEncrypt large data quickly with a local data key, then let a key service protect that data key. S3 encrypts objects this way when you use KMS.
  10. Rotating secrets without an outageRotate a secret by letting the old and new values work together for a while, so no consumer breaks. Dual credentials, short-lived certificates and fresh reads make rotation routine.
  11. Detecting and responding to leaked secretsScan for leaked secrets at commit time, in CI, across git history and at runtime. When one leaks, rotate first and clean history second.
  12. Cloud cost and FinOpsWhere cloud money goes, the levers that usually cut a bill, and how engineers own unit cost without waiting for a finance spreadsheet. Start here for the map of cost pages.
  13. Where a cloud bill goesThe few categories that make up most of a cloud bill, a worked example that cuts $72k to about $43k, and the habit of reading the bill by service and tag.
  14. Pricing models and commitmentsThe three ways to buy the same virtual machine, on-demand, committed and spot, with a worked fleet example that shows how a mix beats any single model.
  15. Hidden network and logging costsThe line items nobody provisions on purpose, NAT gateway processing, cross-zone traffic, internet egress and log ingest, with rough rates and the arithmetic behind each one.
  16. Tagging and unit costHow tags assign cloud spend to owners, how showback and chargeback differ, how unit cost ties the bill to a product metric, and the review habits that keep cost under control.
  17. GPU infrastructureProvisioning and scheduling GPUs for training and inference, covering hardware and fabrics, Kubernetes device plugins, gang scheduling, failures and checkpoints. Start here for the map of GPU pages.
  18. GPU hardware and interconnectsThe three things to check before buying or renting GPUs, memory size, the link between GPUs inside a node, and the network between nodes, with a memory check for a 70B model.
  19. Gang scheduling and Slurm vs KubernetesWhy a multi-GPU job must start all of its workers together, how a default scheduler can deadlock it, and how Kueue, Volcano and Slurm solve the problem differently.
  20. GPU failures and checkpointsHow to detect failing GPUs with XID errors and DCGM, drain and replace bad nodes, and use checkpoints so a card failure costs minutes of training, not days.

Machine learning

From the math you need to training transformers: classical ML, deep learning, LLM pretraining and post-training.

Math toolkit

  1. Vectors and dot productsA vector is a list of numbers. The dot product measures how well two vectors line up, and it drives similarity, retrieval and linear models.
  2. Matrices and matrix multiplicationA matrix maps vectors to new vectors. Matrix multiplication is the operation behind every linear layer, and its shape rule catches most bugs.
  3. Linear algebra for MLVectors, matrices and matrix multiplication, the language every model is written in. Learn the shapes, then batching, FLOP counts and low-rank tricks.
  4. Batching and broadcastingStacking examples into a matrix turns many small multiplies into one large one, which is what GPUs are built for. Broadcasting adds biases and masks, and it fails silently when shapes line up by accident.
  5. Eigenvectors, SVD and low-rank approximationEigenvectors are directions a matrix only stretches. The SVD extends the idea to any matrix, and keeping its strongest parts gives a compact approximation that powers PCA and LoRA.
  6. FLOP counting for matrix multipliesA FLOP is one floating-point operation. Counting the FLOPs of a matrix multiply, using 2mnk, lets you estimate the cost of a layer, a model or a training run before you run it.
  7. Orthogonality and weight initializationOrthogonal vectors point in independent directions. The scale of random starting weights decides whether a signal vanishes or explodes as it passes through layers.
  8. Derivatives as slopesA derivative is the slope of a function at one point, and you can estimate it by nudging the input. Autograd computes the exact value, and finite differences check it.
  9. Partial derivatives and gradientsA partial derivative is the slope along one input while the others stay fixed. The gradient collects all of them into one vector that points uphill.
  10. Calculus and gradientsDerivatives, partial derivatives, the chain rule and gradients. This is the math that turns a loss number into parameter updates.
  11. The chain rule on a graphWhen one quantity depends on another through a chain of operations, their slopes multiply. Applied on a graph, the chain rule gives every gradient in one backward pass.
  12. Jacobians and vector-Jacobian productsWhen a layer maps a vector to a vector, its slopes form a matrix called the Jacobian. Training only needs one vector times that matrix, which a single backward sweep gives.
  13. Derivatives of common functionsThe slopes of sigmoid, tanh, ReLU, and softmax with cross-entropy. Knowing them by heart makes backprop code easier to read and check.
  14. Second derivatives and curvatureThe second derivative says whether the slope is rising or falling. It describes the curvature of the loss, and curvature decides how large a safe learning step can be.
  15. Random variables and distributionsA random variable is a value you cannot know until you observe it. A distribution says how likely each value is.
  16. Conditional probability and BayesHow learning one fact changes the probability of another, and how Bayes' rule runs that change in reverse. Base rates often matter more than test accuracy.
  17. Probability, statistics and informationThe probability ideas behind ML losses, Bayes updates and honest uncertainty. Start here for the map of the section.
  18. Maximum likelihoodMaximum likelihood picks the parameters that make the data you saw most probable. Squared error and cross-entropy are both this one idea.
  19. Softmax and temperatureSoftmax turns raw model scores into a probability distribution. Temperature controls how sharp or flat that distribution is.
  20. Entropy and cross-entropyEntropy measures how surprising a distribution is on average. Cross-entropy measures how well a model's guesses match reality, and it is the classification loss.
  21. Uncertainty and sample sizeA metric from a finite test set is an estimate, not a fact. The law of large numbers, the central limit theorem and the bootstrap tell you how much to trust it.
  22. Loss functions and loss landscapesA loss function scores how wrong a model is. Its graph over the parameters is the loss landscape. Convex landscapes have one bottom, while deep networks have many valleys and flat saddle regions.
  23. Optimization and gradient descentHow models learn, from loss landscapes and gradient descent to learning rates, mini-batches, momentum and Adam.
  24. Gradient descent and the learning rateGradient descent moves the parameters a small step against the gradient. The learning rate sets the step size, and a wrong value makes training either crawl or blow up.
  25. Mini-batch SGD and batch sizeEstimating the gradient from a random mini-batch is cheap and noisy, and that noise is useful. Batch size trades gradient quality, hardware efficiency and generalization.
  26. Momentum and AdamMomentum averages recent gradients to speed up consistent directions. Adam also tracks each parameter's gradient size, which makes it the default for many neural networks.
  27. Conditioning, warmup and learning-rate schedulesFeatures on very different scales create narrow valleys that slow or break gradient descent. Scaling reshapes the valley, and warmup, decay and early stopping control the learning rate over a run.

Classical machine learning

  1. Overfitting, underfitting and bias–varianceA model that memorizes its training data looks brilliant until it sees new data. A model too simple for the pattern fails on both. Two learning curves tell you which problem you have.
  2. Framing ML problemsLearning types, business framing, data splits, baselines and when not to use ML at all.
  3. Types of learningSupervised learning has labels, unsupervised learning has inputs only, self-supervised learning builds labels from the data, and reinforcement learning learns from rewards. The feedback you have decides which one fits.
  4. Framing a business problem as a predictionTurn a business decision into a prediction target, a label source, the decision it drives and the cost of each kind of error. Write these four sentences before any training.
  5. Train, validation and test splitsFit on train, tune on validation, and report once on test. Random splits leak the future and repeated users, so split the way the model will be used.
  6. Baselines and when not to use MLBuild the simplest predictor that already drives the decision, measure it on the same split, and only then train a model. Sometimes a rule or an LLM call is the better tool.
  7. Linear and logistic regressionThe two models under almost everything in ML. They fit lines, predict probabilities and stay honest through regularization.
  8. Linear regression and least squaresLinear regression predicts a number as a weighted sum of features. Least squares picks the weights that make the squared errors as small as possible.
  9. Logistic regression and log-oddsLogistic regression turns a linear score into a probability with the sigmoid and trains with log loss. Its coefficients read easily as odds multipliers.
  10. Regularization with L1, L2 and elastic netRegularization adds a penalty for large weights. L2 shrinks weights smoothly, L1 drives some to exactly zero, and the penalty strength is tuned on validation data.
  11. Feature scaling and leakage-safe pipelinesLinear models and neural nets need features on comparable scales. Learn the scaling from training data only, and keep the scaler inside a pipeline so serving applies the same transform.
  12. Precision, recall and F1A classifier can make two kinds of mistakes, false alarms and misses. Precision and recall measure each one, and F1 combines them. Accuracy hides both when positives are rare.
  13. Evaluating modelsChoose metrics that match the decision, set thresholds from costs, check calibration, and guard against leakage. A high offline score means little until it holds up on real decisions.
  14. Thresholds and costsA classifier's score becomes a yes or no only after you pick a threshold. The right threshold comes from the costs of each mistake, or from the review capacity you have.
  15. ROC-AUC, PR-AUC and intervalsROC-AUC and PR-AUC summarize a ranking across all thresholds, and they can disagree sharply on rare positives. A bootstrap interval shows how much a single AUC number can move.
  16. CalibrationA calibrated model's probabilities match reality. When it says 0.8, about 80% of those cases are positive. Check with a reliability table before you use scores as probabilities.
  17. Data leakageLeakage is when a model learns from information it would not have at prediction time. Offline scores look excellent, then production falls apart. Here is how to find it.
  18. Scaling and transformsLinear models, nearest neighbors and neural nets are sensitive to the size of numbers. Standardizing, log-transforming skewed values and using ratios puts features on comparable footing.
  19. Encoding categories and textCategories and words become numbers through one-hot columns, hashing and bag-of-words counts. Each method has a cost that shows up with many levels or a large vocabulary.
  20. Features and data preparationTurning raw data into model inputs without leaking the future. Covers scaling, encoding, missing values, time windows, target encoding and class imbalance, and keeping training and serving identical.
  21. Missing values and pipelinesA gap in the data can mean something, so flag it and fill it. The fill value must be learned on training data only, which is why imputers belong inside a pipeline.
  22. Point-in-time featuresA feature is valid only if it could have been computed at prediction time. Time windows are the most common source of leakage, and a parity checklist keeps training and serving identical.
  23. Target encoding without leakageTarget encoding replaces a category with its average label. Computed naively, it memorizes the training labels. Out-of-fold statistics and smoothing make it safe.
  24. Class imbalanceWhen positives are 0.1% to 5% of the data, naive training and a 0.5 threshold both fail. Fix the metric and threshold first, then try class weights, and use resampling only as a last resort.
  25. Trees and gradient boostingDecision trees, random forests and XGBoost/LightGBM: still the best default for tabular data.
  26. Decision treesA tree partitions the feature space with axis-aligned splits. Each split is chosen to purify the child nodes, and depth is the main capacity knob.
  27. Bagging and random forestsBagging trains many trees on bootstrap samples and averages them. A random forest also tries a random subset of features at each split, which decorrelates the trees.
  28. Gradient boostingBoosting adds shallow trees one after another, each fitting the residual of the ensemble so far. Shrinkage and early stopping keep the sum from hugging noise.
  29. XGBoost, LightGBM and CatBoostThree libraries implement gradient boosting with different engineering. LightGBM is a fast default, CatBoost handles categoricals well, and features still matter more than the logo.
  30. Unsupervised learningClustering, dimensionality reduction and anomaly detection without labels.
  31. k-means clusteringk-means assigns each point to the nearest of k centroids, then moves each centroid to the mean of its points. Scale features first, and pick k with a plot plus a use case.
  32. DBSCAN and hierarchical clusteringDBSCAN grows dense regions and labels sparse points as noise. Hierarchical clustering builds a tree of merges you can cut after you see the structure.
  33. PCA for machine learningPCA finds orthogonal directions of maximum variance. Keep the top components to compress, denoise or plot. t-SNE and UMAP are for eyes, not for features.
  34. Anomaly detectionWhen the positive class is "anything weird" and labels are rare, flag outliers with thresholds, isolation forests or reconstruction error. The operating point still needs a human cost.

Deep learning

  1. The neuron and why it needs a nonlinearityA neuron takes a weighted sum of its inputs, adds a bias and applies a nonlinear function. Without the nonlinearity, stacking layers gains nothing.
  2. Neural networksFrom a single neuron to multilayer perceptrons: layers, activations, output heads and how to count parameters.
  3. Multilayer perceptrons and the forward passA multilayer perceptron stacks fully connected layers. Each layer is a matrix multiply plus a bias and an activation, and a whole batch moves through the stack as matrices.
  4. Activation functions and vanishing gradientsActivations decide how each neuron bends. Sigmoid and tanh flatten out and starve deep networks of gradient. ReLU and its relatives keep the signal flowing, with a few failure modes to watch.
  5. Output heads and loss functionsThe last layer and the loss must match the question. Regression uses a linear output with MSE, binary classification uses one logit with binary cross-entropy, and multiclass uses softmax with cross-entropy.
  6. Parameter counting, width and depthCounting parameters tells you the memory cost before training starts. Depth builds layered features, width adds capacity in parallel, and the widest matrix usually dominates both memory and compute.
  7. Backpropagation from scratchHow training computes the gradient of the loss for every weight in one backward pass. Covers computational graphs, reverse-mode autodiff and the bugs that break training.
  8. Computational graphs and reverse modeA forward pass records each operation as a node in a graph. The backward pass walks that graph in reverse and applies one local rule per node, so one sweep gives every gradient.
  9. A tiny autograd classA small Value class that remembers how it was made and runs the local rules backward. It is enough to compute every gradient for a scalar expression.
  10. Gradients for linear layers and softmaxThe backward formulas for a batched linear layer, and why softmax with cross-entropy has the simple gradient "probabilities minus the label."
  11. A worked training stepOne full forward pass, backward pass and weight update for a tiny two-unit network, with every number shown. It ends with a dead ReLU unit that receives no gradient.
  12. PyTorch tensorsTensors carry a shape, a dtype and a device. Views, broadcasting and strides explain most of the surprises you will hit.
  13. PyTorch autogradAutograd records the operations you run and computes gradients with backward(). It is the backprop algorithm, done for you.
  14. PyTorch fundamentalsTensors, autograd, modules, data loaders and a clean training loop. The daily-driver API for deep learning. Start here for the map.
  15. PyTorch modules and datann.Module holds the weights and the forward pass. Dataset and DataLoader feed batches to the model. Both have a few traps.
  16. The PyTorch training loopThe canonical loop repeats zero_grad, forward, loss, backward and step. The order matters, and so does switching between train and eval mode.
  17. Mixed precision and checkpointsMixed precision runs most math in a smaller number format to save memory and time. Checkpoints save enough state to resume a run exactly.
  18. Training neural networks wellOptimizers, schedules, initialization, normalization, regularization and a practical recipe for debugging loss curves.
  19. BatchNorm, LayerNorm and RMSNormNormalization rescales activations so each layer sees well-scaled inputs. BatchNorm uses the batch, LayerNorm uses one example's features, and RMSNorm drops the mean.
  20. Debugging loss curvesCheck loss at initialization, overfit one batch, then read train versus val systematically. Most "broken training" is a shape, LR or data bug, not a new optimizer.
  21. Gradient clipping and accumulationClipping caps rare gradient spikes. Accumulation simulates a larger batch when memory is tight. Get the loss averaging right or the effective learning rate silently changes.
  22. Regularization for neural netsWeight decay, dropout, augmentation and early stopping keep a net from memorizing. AdamW applies decay outside the adaptive update, which is why it replaced Adam plus L2.
  23. Convolution as a sliding filterA convolution slides a small kernel across the input and takes a weighted sum at each position. The same weights are reused everywhere, which keeps the layer small and makes it shift-aware.
  24. Convolutional networksHow CNNs use small shared filters, stacked layers and residual connections to learn images, and why transfer learning from pretrained models is the default.
  25. Conv output shapes and parameter countsKernel size, stride and padding set the output size of a convolution. Counting weights shows where a CNN keeps its parameters, which is often in the last dense layer.
  26. Residual connections and ResNetA residual block adds its input back to its output, so each block only needs to learn a change. That keeps very deep networks trainable and gives gradients a path that skips the weights.
  27. Transfer learning for visionReuse a network trained on a large image dataset and adapt it to your own labels. Early layers carry over, and the head and normalization usually need to change.
  28. Pooling and cheap convolutionsPooling shrinks the grid by taking a summary of each window. 1x1 and depthwise separable convolutions cut the weights and compute a standard convolution would need.
  29. Sequence models and the road to attentionRNNs, LSTMs and seq2seq — the problems they hit, and how attention emerged to fix them before transformers took over.
  30. RNNs and backprop through timeA recurrent net compresses the past into a hidden state and reuses the same weights at every step. Training unrolls that loop and runs ordinary backprop on the chain.
  31. Vanishing gradients and LSTMsUnrolling an RNN multiplies the same Jacobian many times, so gradients vanish or explode. LSTMs and GRUs add gated highways so a signal can travel further.
  32. Seq2seq and the fixed-vector bottleneckEarly translation models encoded a whole sentence into one vector, then decoded from it. Long sentences overflowed that vector, which is the problem attention was invented to fix.
  33. Bahdanau attentionInstead of crushing a sentence into one vector, the decoder looks at all encoder states at each step. That soft alignment is the parent of self-attention.
  34. Embedding lookup tablesAn embedding layer is a learned table with one row per ID. Looking up a row turns a discrete ID into a vector that training can shape.
  35. Embeddings and representation learningLearning vectors that capture meaning, from word2vec and contrastive training to CLIP and retrieval. Start here for the map of embedding concepts.
  36. word2vecword2vec trains one vector per word so that words used in similar contexts end up close together. Skip-gram with negative sampling is the version to know.
  37. Contrastive learning and InfoNCEContrastive learning pulls positive pairs together and pushes negatives apart. InfoNCE is the loss that does it with a softmax over a batch, and temperature controls how sharp it is.
  38. Bi-encoders and cross-encodersA bi-encoder embeds the query and the document separately, so search is fast. A cross-encoder reads them together, so it is more accurate but too slow for the whole corpus.
  39. CLIP and zero-shot classificationCLIP trains an image encoder and a text encoder so matching image and caption pairs land close together. That lets you classify images by writing the class names as text.

Transformers and language models

  1. Softmax and dot productsA dot product scores how well two vectors agree. Softmax turns any list of scores into weights that are positive and sum to 1.
  2. Self-attentionThe operation at the core of every modern language model. Each position builds a weighted average of the others, with weights set by query-key matches.
  3. Queries, keys and valuesSelf-attention scores every pair of positions with a query and a key, then mixes value vectors using those scores. The √d_k factor keeps the softmax well behaved.
  4. Attention masksA mask blocks some query-key pairs before softmax. Causal masks stop a language model from reading future tokens, and padding masks ignore filler in a batch.
  5. Multi-head attentionMulti-head attention runs several attention patterns in parallel, each on a slice of the model width, then merges them with one output matrix.
  6. Attention cost and long contextAttention compares every position with every other, so its cost grows with the square of the sequence length. Here are the numbers and the variants that reduce them.
  7. The transformer architectureEvery component of a modern decoder-only LLM, from embeddings to the LM head, with parameter and FLOP counts.
  8. Embeddings and rotary positionsA token id becomes a vector from a table. Attention has no sense of order unless you inject position, and RoPE does that by rotating queries and keys.
  9. The transformer blockA modern decoder block is pre-norm, attention, residual, pre-norm, MLP, residual. The residual stream is the highway. Sublayers read it and write an additive update.
  10. Transformer parameter and FLOP countsNon-embedding parameters are about 12 L d² for a classic decoder block. Forward FLOPs are about 2N per token. Training is about 6N.
  11. Encoder vs decoder transformersBERT looks both ways, T5 has an encoder and a decoder, and GPT looks only left. Decoder-only won for generative LLMs because one stack serves pretraining, SFT and chat.
  12. TokenizationByte-pair encoding in depth: how vocabularies are built and why tokenization causes strange model behavior.
  13. Why subword tokenizationCharacters make sequences too long. Words make the vocabulary unbounded. Subwords sit in the middle, with a fixed vocabulary and a workable sequence length.
  14. Byte-pair encodingBPE builds a vocabulary by greedily merging the most frequent adjacent pairs, starting from bytes so every Unicode string is representable.
  15. WordPiece, SentencePiece and special tokensWordPiece and Unigram are cousins of BPE with different training criteria. Pre-tokenizers, specials and chat templates decide the actual ID sequence you train on.
  16. Tokenizer pathologiesMany "the model can't spell" demos are tokenizer artifacts. Arithmetic, letter counting, trailing spaces and non-English fertility all come from how text was cut into IDs.
  17. Next-token predictionA language model learns by guessing the next token of real text. The text supplies the label, so no human annotation is needed, and the guesses reward learning grammar, facts and patterns.
  18. Pretraining a language modelData, objective and compute: how a base model is trained on trillions of tokens.
  19. The pretraining data pipelineFrontier pretraining is mostly data engineering. Raw web pages become clean, deduplicated, weighted text through extraction, filtering, deduplication and mixing. The data recipe moves quality as much as the architecture does.
  20. Compute budget and the 6ND ruleA training run costs about six FLOPs per parameter per token. Multiply parameters by tokens, divide by realistic throughput, and you have a napkin estimate in GPU-days.
  21. Training dynamics, schedules and checkpointsA long pretraining run needs a learning-rate schedule, a plan for loss spikes and frequent checkpoints. Warmup, cosine or WSD decay, clipping and rollbacks keep a multi-week run alive.
  22. Evaluating a pretraining runValidation loss shows whether a base model is healthy. Downstream benchmarks add signal but can be noisy or contaminated, so check for leaks and compare runs at equal token budgets.
  23. Scaling lawsHow loss scales with parameters, data and compute, and what Chinchilla and test-time compute changed.
  24. Kaplan power lawsFor transformer language models, loss falls as a smooth power law in parameters, data and compute, which is why labs can budget a large run from smaller ones.
  25. Chinchilla and compute-optimal trainingFor a fixed training FLOP budget, scale parameters and tokens roughly equally. A stuck rule of thumb is about 20 tokens per parameter.
  26. Over-training for inferenceChinchilla minimizes training loss per FLOP. Products often train a smaller model on far more tokens so each served token is cheaper.
  27. Extrapolating scaling lawsLabs fit loss on small proxy runs and aim a large run at a target. The fit is only as good as the match in data, architecture and metrics.
  28. KV cache and attention variantsDuring generation, a model keeps keys and values for every past token. MQA, GQA and MLA shrink that cache, which often limits how many long conversations fit on one GPU.
  29. Modern LLM architecturesMixture of experts, GQA and MLA, long context, state space models and multimodal transformers.
  30. Mixture of expertsA mixture-of-experts layer holds many expert networks and sends each token to only a few of them. Total parameters are large, while the compute per token stays closer to a smaller dense model.
  31. Long-context techniquesLong context needs positions that stretch past the training length, attention that stays affordable, and training data that teaches the model to use distant text. A large context number on a spec sheet guarantees none of these.
  32. State space models and hybridsState space models carry a compact running state along a sequence, so their cost grows linearly with length. Hybrids mix these blocks with occasional attention layers to keep sharp recall.
  33. Multimodal transformersImages and audio become sequences of vectors that share one transformer with text. Early fusion trains them together, while late fusion attaches a vision encoder to a frozen language model.
  34. Families of generative modelsAutoregressive models, VAEs, GANs, diffusion and flow matching all learn to sample new data. They differ in how they generate, and in what they make easy and what they make hard.
  35. How diffusion learns to denoiseA diffusion model learns to undo gradual noise. Training adds noise to clean images and teaches a network to predict that noise. Sampling starts from pure noise and removes a little at a time.
  36. Generative models beyond textHow images, video and speech are generated. Compares the main model families, builds diffusion from first principles, and explains why diffusion inference costs so differently from chat.
  37. Latent diffusion, guidance and conditioningLatent diffusion runs the denoising in a small compressed space, and classifier-free guidance controls how closely the output follows the prompt. Conditioning covers the other inputs a denoiser can take.
  38. Fewer steps, samplers and distillationTextbook diffusion needs hundreds of denoising steps. Better samplers, noise schedules and distilled models produce good images in a few steps, which makes interactive products practical.
  39. Diffusion inference costServing an image model is a different systems problem from serving an LLM. Each image runs the denoiser many times over the whole latent, so the cost is steps times denoiser cost, not one pass per token.
  40. Speech and video generationSpeech and video use the same generative families with different bottlenecks. Speech often becomes discrete audio tokens, and video multiplies the work by the number of frames.

Scale and post-training

  1. Memory math for trainingTraining memory is much more than the weights. Mixed-precision Adam needs about 16 bytes per parameter before any activations, which is why a 7B model needs over 100 GB.
  2. Mixed precisionMixed precision runs most matrix multiplies in 16-bit floats and keeps a 32-bit master copy of the weights. fp16 needs loss scaling to avoid underflow. bf16 usually does not.
  3. Distributed trainingHow to train models too big for one GPU, using data, sharding, tensor, pipeline and context parallelism. Covers the memory math that forces the choice and how to judge the result.
  4. Data parallelism and all-reduceData parallelism gives each GPU a full copy of the model and a different slice of the batch. The GPUs then sum their gradients with an all-reduce, so every copy applies the same update.
  5. ZeRO and FSDPZeRO and FSDP remove the copies that data parallelism makes of optimizer state, gradients and weights. Each GPU keeps only a shard and gathers the rest when a layer needs it.
  6. Tensor, pipeline and context parallelismModel parallelism splits the network itself. Tensor parallelism splits weight matrices, pipeline parallelism splits layers into stages, and context parallelism splits the sequence. Each one needs a different kind of network link.
  7. Planning a training runChoose a parallel layout that fits in memory, use checkpointing and accumulation to fit activations, and judge the result by MFU. Worked through a 70B model on 64 GPUs.
  8. Supervised fine-tuning and PEFTInstruction tuning, chat formats, data quality, LoRA/QLoRA math, and the tooling that turns a base model into a useful assistant.
  9. Instruction tuning and loss maskingSupervised fine-tuning teaches a base model to follow instructions. Gradients should flow through the assistant tokens, not the user's prompt, and train and serve must share one chat template.
  10. LoRALoRA freezes the base weights and trains a pair of thin matrices whose product is the update. Trainable parameters drop by orders of magnitude.
  11. QLoRAQLoRA freezes the base model in 4-bit NormalFloat and trains LoRA adapters in higher precision, which is why a 70B model can be fine-tuned on one 48 GB GPU.
  12. PEFT vs full fine-tuningFull fine-tuning updates every weight. PEFT freezes the trunk and trains a small adapter. Default to LoRA for product work, and go full only when you need a large shift.
  13. MDPs, returns and valuesThe vocabulary of reinforcement learning. An agent acts, the environment returns a state and a reward, and returns and values measure how good those actions are over time.
  14. Reinforcement learning basicsMDPs, returns, policy gradients, advantages and PPO. The minimum RL needed to read modern LLM post-training papers. Start here for the map.
  15. Policy gradients and advantagesPolicy gradients adjust the policy directly. Make the actions that led to high returns more likely. Baselines and advantages cut the noise in that update.
  16. PPO and the KL leashProximal Policy Optimization (PPO) limits how far one update can move the policy. The clipped ratio and a KL penalty keep language models close to where they started.
  17. RL for language modelsHow the RL vocabulary maps onto a language model. The model is the policy, tokens are actions, and a completion is the episode. Credit assignment and exploration are the hard parts.
  18. Preference pairs and the Bradley-Terry reward modelPeople find it easier to compare two answers than to write a perfect one. Preference pairs train a reward model with a simple logistic loss.
  19. RLHF and preference optimizationHow language models learn from human and AI preferences. Supervised fine-tuning, a reward model, a KL-leashed RL step, and DPO as a single-loss alternative. Start here for the map.
  20. The KL leash in RLHFThe third RLHF stage trains the policy against a reward model, with a KL penalty to the supervised model. The penalty weight decides whether the model improves or games its reward.
  21. Direct preference optimization (DPO)DPO optimizes the same KL-constrained goal as RLHF with a single loss on preference pairs. It needs no separate reward model and no sampling loop.
  22. RLAIF and Constitutional AIRLAIF uses a model's judgments in place of human labels, and Constitutional AI makes the rubric an explicit list of principles. Both scale feedback, and both need human checks.
  23. Chain of thought and test-time computeChain of thought is intermediate reasoning written before the answer. Test-time compute is the extra inference work spent on a query, and it can buy accuracy on hard problems.
  24. Verifiable rewardsReasoning RL works best when a checker can say whether an answer is right. Exact answers, unit tests and proof checkers give the cleanest reward a language model can get.
  25. RL for reasoning modelsHow reasoning models learn to think before they answer, using verifiable rewards, GRPO and test-time compute. Start here for the map.
  26. GRPOGroup Relative Policy Optimization samples several completions per prompt and uses their average reward as the baseline. It drops the value network that PPO needs.
  27. The R1 training recipeDeepSeek-R1 combined pure RL, cold-start supervised data, more RL and distillation. The transferable pattern is verifiable RL, light supervised polish, then distill to smaller models.
  28. Process vs outcome rewardsAn outcome reward scores only the final answer. A process reward scores each step. Outcomes are simpler and the default, while process rewards give finer credit at a higher labeling cost.

Applied ML systems

  1. Implicit feedback and objectivesMost recommender training data is implicit, such as clicks, watches and skips. Those signals are plentiful but noisy, and the objective you choose decides what the system learns.
  2. Collaborative filtering and matrix factorizationCollaborative filtering recommends items from patterns of who interacted with what. Matrix factorization learns a short vector for each user and item, so a dot product predicts affinity.
  3. Recommender systemsHow feeds and stores pick a short list from millions of items. A cheap retrieval step, a careful ranker and business rules. Start here for the map.
  4. Two-tower retrievalA two-tower model embeds users and items separately, so item vectors can be indexed in advance and a nearest-neighbor search finds candidates fast.
  5. Cold start, popularity bias and feedback loopsNew users and items have no history, popular items crowd out the rest, and the model's own recommendations shape its next training data. Each has a practical fix.
  6. YouTube recommendations, 2016A case study of the 2016 YouTube paper. Two stages, candidate generation then ranking, with embeddings for videos and a watch-time objective. It is still the clearest public blueprint.
  7. Learning to rank basicsLearning to rank trains a model to order a list. Pointwise, pairwise and listwise methods differ in what the loss compares, not in how you serve the sorted list.
  8. Ranking metricsNDCG, MRR, MAP and recall at k measure how good an order is. Work one NDCG example by hand and the formula stops being abstract.
  9. Ranking and learning to rankSecond-stage ranking for search, feeds and ads. Learning to rank, ranking metrics, LambdaMART, CTR models, position bias and calibration. Start here for the map.
  10. LambdaMARTLambdaMART combines boosted trees with pairwise gradients scaled by how much a swap would change NDCG. It is the workhorse of listwise ranking.
  11. CTR predictionClick-through rate models estimate the chance a user clicks an item. The model family grew from logistic regression to embedding-and-interaction networks.
  12. Position biasUsers click the top result more often even when it is worse. Raw click logs therefore reflect the old ranking, and inverse propensity weighting, randomization and logging can correct for it.
  13. Calibration and multi-objective rankingA calibrated score matches how often the event happens. Multi-objective ranking blends several predicted outcomes into one score, and the weights are product decisions.
  14. The ML system lifecycleA deployed model is a loop of data, features, training, evaluation, serving and monitoring. Each stage produces an artifact you can record, check and roll back.
  15. ML in productionFeature stores, training-serving skew, monitoring for drift and the lifecycle of a deployed model.
  16. Batch vs online inferenceBatch scoring writes predictions ahead of time. Online scoring answers while a user waits. Choose by how fresh the features must be and how much latency the product can afford.
  17. Training-serving skew and point-in-time featuresTraining-serving skew happens when the code or data used to predict in production differs from the code or data used to train. Point-in-time joins and shared feature definitions prevent most of it.
  18. Monitoring drift and delayed labelsProduction models fail quietly. Watch inputs, predictions and business numbers now, and confirm model quality later with labels that arrive days or weeks after each decision.
  19. Deploying models safelyRoll out a new model in stages, shadow then canary then A/B test. Gate each step on offline slices and product metrics, and keep the previous version one step away.

AI engineering

Building reliable products on large language models: prompting, context, retrieval, agents, evals and production.

LLM fundamentals

  1. The LLM landscape and model choiceFrontier vs open-weight models, reasoning models, and a process for picking one with your own evals.
  2. Frontier vs open-weightFrontier models arrive through a vendor API. Open-weight models are downloadable parameters, not automatically open source.
  3. Reasoning modelsExtra inference-time compute, often a hidden chain of thought, billed as output tokens. Use them on the hard slice, not on "summarize this email."
  4. Choosing a modelProve the product on a strong model, freeze an eval, then step down the ladder. Your suite decides, not a leaderboard.
  5. Why benchmarks lieLeaderboards answer "is this model generally strong?" They do not answer whether it extracts your invoice fields at 99% with your schema.
  6. What a token isModels read integer IDs, not words. The same sentence can be a different number of tokens on two APIs.
  7. Tokenization quirksLetter-counting, long numbers and some languages fail because the model sees pieces, not characters. Measure real payloads.
  8. Tokens and context windowsHow text becomes tokens, what that means for cost and limits, and how to budget a context window.
  9. Context windowsThe window is input plus output. Everything you send — instructions, tools, history, retrieved docs — shares one budget.
  10. Token pricingBills are USD per million tokens. Output is often 3 to 5 times input. Caching a stable prefix can cut the input line.
  11. Budgeting a context windowTreat the window like RAM for one request. Cap history, drop old tool dumps and leave room for the answer.
  12. The next-token loopA decoder-only model is a function from tokens so far to the next token. Products wrap that loop with different prompts and stop rules.
  13. Attention for buildersEach position mixes earlier positions with a weighted average. Anything in the context can influence the next token.
  14. How transformers work (practitioner view)Just enough transformer internals to predict model behavior — attention, context, and why the KV cache shapes pricing.
  15. Lost in the middleAttention can look anywhere earlier in the context, but quality is often best at the edges and worse in the middle of a long prompt.
  16. Prefill, decode and the KV cacheOne parallel pass over the prompt, then one token at a time. Reusing keys and values is why a stable prefix saves money.
  17. Weights vs context vs toolsGrammar lives in the weights. Private docs live in the prompt. Live account state lives in tools. Hallucinations thrive when you pick the wrong row.
  18. Sampling and decodingTemperature, top-p, top-k and other knobs that turn logits into text, and when to touch them.
  19. TemperatureTemperature divides logits before softmax. Low values peak on the top token. High values flatten the distribution and let rare tokens through.
  20. Top-k, top-p and min-pTruncation deletes the long tail of the vocabulary, then renormalizes. Top-p adapts to confidence. Top-k does not.
  21. Decoding controlsMax tokens, stop sequences, penalties and logprobs. Temperature 0 removes sampling noise. It does not make two API calls bit-identical.
  22. Self-consistency and best-of-nSample several times and pick with a vote or a verifier. You buy robustness with n times the output tokens, and only when the answer is checkable.
  23. Working with LLM APIsMessages, roles, streaming, stop reasons, errors, rate limits and batch APIs. The plumbing of every LLM app.
  24. Chat messages and rolesEach call resends the full conversation. Roles mark who said what. Pin the model id and version the system prompt.
  25. Streaming and stop reasonsTokens arrive as events. The model always stops for a reason. Branch on that reason. Do not assume success.
  26. Retries and rate limitsRetry 429 and 5xx with exponential backoff and jitter. Never blindly retry a tool that already had a side effect.
  27. Batch LLM APIsSubmit a file of requests, get results hours later, typically at about half the sync token price. For overnight work, not chat.

Prompting and context

  1. PromptingClear instructions, examples, structure and chain-of-thought: prompting as specification writing.
  2. Prompts as specificationsA prompt is a written spec: goal, inputs, constraints, output shape and success criteria. Vague asks produce vague answers.
  3. Prompt structureWhere each part of a prompt goes: durable rules in the system message, per-request data in labeled sections, and the question last.
  4. Few-shot examplesInput and output demonstrations that teach format and judgment, why diversity beats volume, and when examples over-constrain.
  5. Chain-of-thought and extended thinkingWhen asking the model to reason step by step helps, how native thinking differs from prompted reasoning, and how to keep the reasoning out of the answer.
  6. Structured outputsGetting reliable JSON: schemas, constrained decoding, validation and repair.
  7. Constrained decodingHow a schema can be enforced during generation by masking tokens that would break it, and what that guarantee does not cover.
  8. Schema designHow to write JSON schemas the model fills in correctly, using flat shapes, enums, explicit nulls and field descriptions as instructions.
  9. Validation and repairWhy every structured output still needs code-level validation, how a bounded repair loop works, and how to handle refusals and truncated output.
  10. Field-level evals for extractionHow to score structured outputs field by field, with worked arithmetic, and why "100% valid JSON" can hide wrong invoice IDs.
  11. Context engineeringDeciding what goes into the context window: ordering, compaction, memory and long context vs retrieval.
  12. Context orderingPut a stable, cacheable prefix first and the live question last, so quality and prompt caches both win.
  13. Compaction and just-in-time retrievalSummarize old turns, trim tool dumps and re-fetch full payloads only when the agent can navigate to them.
  14. Long context vs RAGPaste the working set when it fits. Retrieve when the corpus is large and only a few passages matter. Many products do both.
  15. Multimodal inputs and outputsImages, PDFs, audio and video in LLM apps: what works, what it costs and how to evaluate it.
  16. Vision tokens and costProviders bill images as tokens after a resize. Resolution, crops and detail settings decide both the invoice and whether small text is readable.
  17. Document understandingPDFs, tables and charts: prefer a native text layer when it exists, and use vision for the parts text cannot see.
  18. Audio and videoAudio is either speech-to-text plus a text model, or native audio tokens. Video is sampled frames. Cost scales with duration and resolution.
  19. Multimodal evalsGrade real photos and screenshots, not clean demos. Schema validity is not a correct total, and production lighting is part of the fixture.

Retrieval

  1. Embeddings and similarityTurning text into vectors, measuring similarity, and choosing an embedding model.
  2. Vector similarity: cosine, dot product and distanceHow to compare two embeddings with cosine, dot product or Euclidean distance, and why normalizing vectors makes them agree.
  3. Bi-encoders and cross-encodersWhy retrieval encodes the query and documents separately, why reranking reads them together, and how the two work in sequence.
  4. Choosing an embedding modelHow to pick an embedding model by running a small bake-off on your own documents, weighing quality, dimension, input length, language and cost.
  5. Embeddings beyond searchClustering, near-duplicates, cheap classifiers and recommendations all use the same vectors you already store for retrieval.
  6. Exact vs approximate searchExact kNN checks every vector. ANN misses a few neighbors on purpose so a 10-million-row index can answer in milliseconds.
  7. Vector search and vector databasesApproximate nearest neighbor algorithms (HNSW, IVF, PQ) and choosing between pgvector and dedicated stores.
  8. HNSWA layered graph. Search walks sparse highways first, then a dense bottom layer. Tune M, efConstruction and efSearch.
  9. IVF, PQ and DiskANNCluster, compress or spill to SSD when a pure HNSW graph no longer fits in RAM.
  10. Filtered vector searchAlmost every real query is nearest neighbors among documents the caller may see. Filter at retrieval, never in the prompt.
  11. Choosing a vector storeStart with pgvector when Postgres is enough. Move to a dedicated store for RAM, filters, hybrid search or ops pain.
  12. Retrieval-augmented generationThe core RAG pipeline: ingest, chunk, embed, retrieve, generate with citations.
  13. The RAG ingestion pipelineThe offline half of RAG, from parsing documents to syncing edits, deletes and access rules, so the index never drifts from the source.
  14. Chunking for RAGHow to cut documents into passages for retrieval, why naive character cuts break facts, and how size, overlap and context prefixes change what can be found.
  15. Grounded answers with citationsThe query-time path of RAG, from question to top-k chunks to a grounded prompt that refuses when evidence is missing and cites sources that code can check.
  16. RAG failure modes and debuggingFive ways a RAG answer goes wrong, how to tell them apart from one failing trace, and the first fix for each.
  17. Retrieval metricsRecall@k, MRR and nDCG tell you whether the right chunk was found, before you blame the prompt or the model.
  18. Advanced RAGHybrid search, reranking, query rewriting, contextual retrieval and measuring retrieval quality.
  19. Hybrid searchRun BM25 for exact tokens and dense vectors for paraphrases, then fuse the two ranked lists with reciprocal rank fusion.
  20. RerankingRetrieve a wide pool with a bi-encoder, then score query and passage together with a cross-encoder so the prompt gets the best 5 to 10.
  21. Query transformationRewrite, split, expand or hypothesise a search query before retrieval, so messy questions still hit the right chunks.
  22. Contextual retrievalSituate every chunk with document and section context before you embed it, so "Revenue grew 3%" still names the company.

Agents and tools

  1. The tool loopThe model emits a structured call. Your runtime runs it, or refuses it, and sends the result back. Bound the loop.
  2. Tool use and function callingLetting models call your code: tool schemas, the tool loop, and designing tools models use well.
  3. Tool schemasNames, descriptions and JSON Schema are prompts. Vague contracts are the most common cause of the wrong tool.
  4. Designing toolsFewer, higher-level tools with compact, paginated outputs beat a pile of micro-endpoints the model must stitch together.
  5. Safe tool executionYour executor owns auth, idempotency and approvals. The model proposes. It does not authorize.
  6. Evaluating tool useScore tool choice, arguments and error recovery — not only the final sentence. Fifty good cases beat a thousand random chats.
  7. Agents and agent architecturesWorkflows vs agents, the core patterns, ReAct loops, planning and knowing when to stop.
  8. Workflow patternsChaining, routing, parallelization, orchestrator-workers and evaluator-optimizer cover most production systems before you need an open agent loop.
  9. The ReAct loopReason, act, observe, repeat, with hard stop conditions so the loop cannot run forever.
  10. Agent planningWrite a plan first when you need approval or a large decomposition. Interleave the next action when the environment keeps surprising you.
  11. Agent autonomy and stoppingAutonomy is a dial. Pick a level, cap the loop, verify writes with external signals and put a human on irreversible actions.
  12. Model Context Protocol (MCP)The open standard for connecting models to tools and data: architecture, primitives, transports and security.
  13. Hosts, clients and serversMCP turns N times M custom adapters into N plus M. A host owns the app, a client talks to one server, and a server exposes tools and data.
  14. Tools, resources and promptsMCP servers expose three primitives: tools the model calls, resources the app can read, and prompt templates the user can insert.
  15. stdio and HTTP transportsTwo ways to move MCP messages: a local subprocess over stdin/stdout, or a remote HTTP server with OAuth.
  16. MCP securityA connected server can steer the model, act with the user's token and read tool results that contain attacker text. Pin, scope and enforce policy in code.
  17. Multi-agent systemsWhen multiple agents beat one: orchestrator-subagent designs, context isolation and the cost of coordination.
  18. Orchestrator and workersOne planner writes a task list. Workers run those tasks in parallel with their own tools, then return condensed results.
  19. Context isolationEach worker should see only its brief and its own tool results. Summaries at the boundary must keep citations and caveats, not raw HTML.
  20. Coordination and costExtra agents multiply tokens and create new failure modes. Budget the fan-out, then watch duplication, conflicting edits and lossy merges.
  21. Evaluating multi-agent systemsScore the final artifact, then slice failures into coverage, handoff fidelity, efficiency and contradictions.
  22. Agent infrastructureRunning agents for real: sandboxes, durable execution, state, permissions and long-running tasks.
  23. Agent sandboxesCode, browsers and filesystems run somewhere else, with no production secrets, capped resources and an allowlisted network.
  24. Durable agent executionRecord each completed step so a crash, a deploy or an overnight approval resumes instead of starting over.
  25. Agent state and memorySplit conversation, artifacts, working memory and run control. Persist after each step. Reload a hot working set, not the whole transcript.
  26. Agent permissions and auditAuthorize every tool against the real user, keep policy in code next to the executor and log who approved each side effect.
  27. Cascaded vs speech-to-speechSTT then LLM then TTS gives you control. A native speech model gives you feel. Many teams ship one and prototype the other.
  28. Voice agentsReal-time speech AI: STT→LLM→TTS pipelines vs speech-to-speech models, latency budgets and turn-taking.
  29. Voice latency budgetsHumans notice gaps. Target roughly 500 to 800 ms from end of speech to first audio, and get there by overlapping stages.
  30. Turn-taking and barge-inVAD finds speech. Endpointing decides the turn is over. Barge-in must stop TTS and cancel the old generation immediately.
  31. Streaming the voice pipelineOverlap STT partials, LLM tokens and TTS frames. Flush on clause boundaries. Speculate only on safe, read-only prep.
  32. Evaluating voice agentsScore latency, barge-in, ASR impact and task success together. A scripted call suite belongs in CI next to text evals.

Quality and safety

  1. Evaluating LLM appsA fixed set of inputs plus a way to score outputs, so you can tell whether a change helped.
  2. Error analysis and failure taxonomiesRead real traces, name what went wrong, and group the notes into categories. Those categories decide which evals you write.
  3. Eval gradersA grader is the rule that decides whether one output passes. Use the cheapest grader that catches the failure reliably.
  4. Offline and online evalsOffline evals run on a fixed dataset before you ship. Online evals score live traffic. You need both, and every run should record its versions.
  5. Eval noise and sample sizeA pass rate is a proportion, and proportions wobble. Learn how big the wobble is at your sample size before you celebrate a small gain.
  6. LLM-as-a-judgeUsing models to grade models: when a judge is the right grader, how to design one, and why an uncalibrated judge is a vibes function.
  7. Judge designAsk for a short critique and a binary verdict. One criterion per judge. Make the verdict parseable.
  8. Calibrating LLM judgesLabel 100 to 200 examples with a human, then treat FAIL-detection as a classification problem. Fix the rubric before you swap models.
  9. Judge biasesJudges prefer some answers for reasons that are not quality. Position, verbosity and self-preference are the ones to plan for.
  10. Pairwise judge comparisonsWhen comparing two prompts or models, ask which answer is better on one criterion, then swap the order and only count consistent winners.
  11. Guardrails, security and safetyPrompt injection is unsolved. Cut blast radius with architecture, not a classifier that claims to detect it.
  12. Prompt injectionUntrusted text that the model treats as instructions. Direct injection is in the user message. Indirect injection hides in fetched content.
  13. The lethal trifectaAn agent becomes dangerous when it has private data, untrusted content and a way to send data out. Break any one leg.
  14. Safety controlsPrefer mechanical enforcement over asking the model to behave. Least privilege, confirmation, allowlists and quarantine.
  15. Red teaming LLM appsWrite attacker stories from the trifecta, turn them into eval cases, and retest whenever tools or data sources expand.
  16. Fine-tuning vs prompting vs RAGWhen to change weights instead of the prompt or the retrieved context, and how to decide with evals.
  17. Kinds of fine-tuningSFT teaches an input-to-output map. Preference methods rank answers. Distillation copies a teacher. LoRA makes the update cheap.
  18. Fine-tuning dataQuality and coverage of the examples dominate the training run. Match production inputs and freeze a held-out eval first.
  19. Serving fine-tunesManaged APIs trade control for speed. Open-weight LoRA trades ops for isolation. Plan for base-model upgrades either way.

Production and case studies

  1. Latency and cost optimizationStreaming, routing, cascades, batching and parallelism: making LLM apps fast and affordable without dropping quality.
  2. LLM latency metricsMeasure TTFT, tokens per second, end-to-end latency and cost per successful task before you change the prompt or the model.
  3. Streaming for perceived latencySend tokens as they arrive. The model still takes the same time to finish, but users feel time-to-first-token instead of a blank wait.
  4. Reducing LLM tokensTokens are both latency and money. Shorten the prompt, cap the output, and drop unused tools before you reach for a new model.
  5. Model routing and cascadesSend easy work to a small model and hard work to a frontier model. A cascade tries cheap first and escalates when a check fails.
  6. Caching for LLM appsPrompt caching, exact-match and semantic caches: where each fits, and how to avoid serving the wrong answer.
  7. Prompt cachingProviders reuse KV state for an identical prompt prefix. Put static content first. A one-token change early in the prompt misses everything after it.
  8. Exact-match LLM cacheWhen the full request fingerprint is identical and sampling is deterministic enough, store the completion and skip the model.
  9. Semantic LLM cacheEmbed the query, reuse a prior answer if it is close enough. High leverage on public FAQ. Dangerous on anything personal or policy-sharp.
  10. Cache invalidationEvery LLM cache key needs model, prompt, tools, knowledge and tenant. Prefer versioned keys over hoping a TTL saves you.
  11. LLMOps and observabilityTracing, prompt management, feedback loops and online monitoring for LLM applications.
  12. Tracing and logging LLM callsWhat to record for each model call, how to build a trace tree for a whole request, and how to keep the logs private.
  13. Prompt versioningHow to version prompts like code, where to keep them, and the rules that let you trace any production answer back to the exact prompt that produced it.
  14. Online evals, feedback and alertsHow to check live traffic with cheap code checks, sampled judges and user feedback, and which signals should page someone when quality drops.
  15. LLM gatewaysThe proxy between your apps and model providers that centralizes keys, quotas, retries, fallbacks and spend, and the rules that keep retries and fallbacks safe.
  16. Designing AI product experiencesInteraction design for probabilistic systems: trust, control, failure states and human-in-the-loop.
  17. AI interaction modelsChat, inline assist, copilot, autonomous agent or background batch. Pick the shape that matches stakes and how structured the task is.
  18. Calibrated trustUsers over-trust fluent systems. Show limits, citations and honest uncertainty so they rely on the product when it is strong and check it when it is weak.
  19. Designing for AI failureAssume the model will be wrong. Make correction, undo, partial success and empty retrieval first-class screens, not polish.
  20. Human-in-the-loopA human approves, edits or takes over before the system proceeds. Use it for irreversible actions and the long tail your evals have not made safe.
  21. Support agent requirementsWrite autonomy, safety, handoff, freshness, latency and cost as constraints. Define in-scope and what "resolved" means.
  22. Case study: customer support agentEnd-to-end: RAG over help docs, account tools, escalation to humans, evals and cost.
  23. Support agent architectureA router, a read-only RAG answerer, a bounded tool agent and a handoff pack. Not one free-roaming agent on day one.
  24. Support tools and permissionsSession-scoped reads, propose-then-confirm for money, humans for irreversible actions. Bind identity from the session.
  25. Support evals and costLabel 200 to 500 real tickets. Gate on policy and never-call cases. The cost model only counts if CSAT holds.
  26. Support agent rolloutShadow, then FAQ, then reads, then confirmed writes. Keep a kill switch. Staff the desktop so humans improve the bot.
  27. Case study: coding agentHow coding agents work: repo context, edit tools, test loops, sandboxes and review.
  28. Agentic code searchGrep, glob and targeted reads beat dumping a repo or trusting a nearest-neighbor hit that looks relevant and is the wrong module.
  29. Patch edit toolsUnique-match search-and-replace with a clear error beats sloppy full-file rewrites for most coding-agent edits.
  30. Coding agent evalsSWE-bench compares scaffolds. Your own suite catches monorepo conventions, test gaming and over-eager edits.
  31. Coding agent reviewForce a branch and a PR. Humans own architecture, security and test integrity. A fresh-context reviewer catches what the author agent missed.
  32. Case study: document extraction pipelineStructured data from millions of documents: schemas, batch APIs, validation and human review.
  33. Extraction schemas and validatorsA versioned JSON schema plus business-rule checks in code. Schema-valid is not the same as a total that adds up.
  34. Extraction tiering and batch costAt a million documents a month, batch APIs and a cheap-to-frontier model ladder matter more than a single sync call.
  35. Extraction confidence and reviewModels are not calibrated out of the box. Route on validators, agreement and a plot you measured, then make correction cheap.
  36. Extraction reprocessingTreat every extract as a pure function of bytes, schema and extractor version. Re-runs write a new row. Never overwrite a human-approved value in silence.

Inference engineering

Serving models fast and cheaply on GPUs: hardware, the math of LLM inference, optimizations, serving systems and economics.

GPU fundamentals

  1. The latency-throughput-cost triangleEvery serving choice trades user wait, tokens per GPU, and dollars per million tokens. Batch size is the main dial.
  2. The inference stackServing is layers from GPU to gateway. Wins and outages live at one layer. Blaming the GPU for a Python launch tax is a category error.
  3. What inference engineering isTurning a checkpoint into a fast, cheap, reliable service. Start here for the latency-throughput-cost map and the serving stack.
  4. Warps and SIMTGPUs schedule work in warps of 32 threads that share one instruction. Divergence and occupancy decide how much of the chip you actually use.
  5. GPU memory hierarchyRegisters, shared memory, L2 and HBM, with H100 numbers, and why inference runs out of memory more often than it runs out of FLOPs.
  6. GPU architectureSMs, warps, tensor cores and the memory hierarchy from registers to HBM, with H100/B200 numbers.
  7. Tensor cores and peak FLOPsTensor cores do the matrix math that LLMs need. Use the dense peak, then the ridge point, to see whether a kernel can ever hit that number.
  8. GPU interconnectsNVLink, PCIe and InfiniBand set how fast GPUs can share work. Keep latency-sensitive collectives inside an NVLink domain.
  9. Kernels, grids and threadsA kernel is a device function launched as a grid of blocks of threads, each computing a different index.
  10. Coalescing and shared-memory tilingAdjacent warp loads become few HBM transactions, and tiles in shared memory let you reuse each loaded byte.
  11. The CUDA programming modelKernels, grids, blocks and threads, memory coalescing and occupancy, plus Triton as the higher-level path.
  12. Occupancy and launch overheadResident warps hide HBM latency, and a 5–10 µs launch tax wrecks decode when every op is tiny.
  13. Triton and when to write a kernelPrefer library GEMMs and attention. Use Triton for fusions. Drop to raw CUDA only when you need control the compiler cannot express.
  14. Arithmetic intensity and the ridge pointIntensity is FLOPs per byte moved from HBM. The ridge is where the bandwidth slope meets the compute roof.
  15. Roofline examplesElementwise add sits far left of the ridge. A large square matmul sits to the right. Tiny GEMMs fall back onto the bandwidth slope.
  16. The roofline model and arithmetic intensityPredicting performance from FLOPs and bytes: compute-bound vs memory-bound. Start here for the map, then work the examples.
  17. Decode batch size on the rooflineDecode intensity scales with how many sequences share one weight read. Batch-1 crawls the bandwidth slope. Batching walks you toward the roof.
  18. Communication rooflinesMulti-GPU serving has a second roof. When collective time exceeds local compute, more tensor-parallel GPUs can slow tokens/s.
  19. Launch overheadDecode runs many short kernels. Python dispatch and kernel launches can cost a millisecond per token before any matmul starts.
  20. Kernel fusionCombine several ops into one GPU kernel so intermediates stay in registers or shared memory instead of round-tripping through HBM.
  21. Kernels, fusion and compilationKernel fusion, torch.compile, CUDA graphs and the libraries that make models fast.
  22. CUDA graphsRecord a fixed sequence of GPU launches once and replay it every token, so the CPU is no longer the metronome for decode.
  23. torch.compilePyTorch traces a model into a graph, fuses the glue with Inductor and Triton, and can wrap the result in CUDA graphs. Dynamic shapes are the catch.

Anatomy of LLM inference

  1. Inference arithmeticNapkin math for any model on any GPU: weight memory, decode and prefill time, batch limits and KV budget.
  2. Weight memoryHow many bytes a model's weights take, from parameter count times bytes per parameter, and why precision decides what fits.
  3. Decode latency floorWhy one generated token costs roughly the weight bytes divided by memory bandwidth, and how that gives a tokens-per-second ceiling.
  4. Prefill timeHow long the prompt takes to process before the first token, estimated from parameter count, prompt length and achieved FLOPs.
  5. Batch ceilingHow batching raises decode throughput until compute runs out or the KV cache fills, and how to estimate the concurrency limit.
  6. The prefill phasePrefill reads the whole prompt in parallel, writes the KV cache, and emits the first token. It is usually compute-bound and sets time to first token.
  7. The decode phaseDecode emits one token per step, re-reads the weights and the growing KV cache, and is usually memory-bandwidth-bound. It sets time per output token.
  8. Prefill vs decodeThe two phases of LLM inference. Prefill is compute-bound and sets TTFT. Decode is memory-bound and sets TPOT. They interfere when they share a GPU.
  9. Inference workload shapesChat, RAG, reasoning and agents sit in different corners of the input-versus-output plane. Each corner wants a different capacity plan.
  10. Why the KV cache existsEach decode step would rebuild keys and values for every past token. The cache stores them once and appends one new column per step.
  11. KV bytes per tokenEach token stores K and V on every layer. Llama-3-70B in BF16 is about 320 KB per token, so a long chat can rival a small model.
  12. The KV cacheWhat keys and values the cache stores, how big it gets per token, and how GQA, MLA, quantization and offloading shrink it.
  13. GQA, MQA and MLASharing or compressing KV heads shrinks the cache and the bytes moved each decode step, which is usually a better first fix than quantizing a bloated MHA cache.
  14. KV offloadingHot tokens stay in HBM. Warm prefixes spill to CPU or SSD. Hits beat recomputing prefill. Cold million-token decode still pays the bandwidth.
  15. TTFT and TPOTTime to first token is queue plus prefill. Time per output token is decode smoothness. End-to-end latency is those two put together.
  16. GoodputThroughput is tokens you completed. Goodput is tokens that also met the latency contract. Capacity planning uses the second number.
  17. Inference metricsTTFT, TPOT/ITL, end-to-end latency, throughput and goodput — definitions, percentiles and how they trade off as concurrency rises.
  18. The latency-throughput curveAs concurrency rises, tokens per second climb then flatten, per-user speed falls, and tail TTFT spikes. The SLO knee is the operating point.
  19. Inference SLOsWrite dual thresholds for TTFT and TPOT, name the workload next to them, and autoscale on goodput headroom rather than GPU utilization.

Core optimizations

  1. Why batching speeds up decodeOne weight read from HBM can serve many sequences, so larger batches raise arithmetic intensity.
  2. Static and dynamic batchingPadding and tail idle waste GPU slots, and a timeout still locks the batch until the longest sequence ends.
  3. Batching and continuous batchingWhy batching amortizes weight reads, how static and dynamic batching waste GPU time, and how iteration-level continuous batching plus chunked prefill unlocks 10–20× throughput.
  4. Continuous batchingReschedule every model forward so finished sequences free a slot on the next step, not when the longest one ends.
  5. Chunked prefillSplit a long prompt into token chunks so decode sequences keep making progress every iteration.
  6. Why reserved KV memory wastes the GPUReserving one contiguous block of KV cache for the longest allowed sequence leaves most of it empty, and scatters the free space into pieces too small to use.
  7. PagedAttention and KV memory managementHow reserving contiguous max-length KV wastes most of GPU memory, and how fixed-size KV blocks with block tables eliminate fragmentation and unlock sharing.
  8. Block tables and KV pagesPagedAttention cuts the KV cache into fixed-size blocks and maps each sequence's tokens to them with a block table. Blocks are allocated only as a sequence grows.
  9. Copy-on-write KV sharingSequences that share a prompt can point at the same KV blocks. Only the block that diverges is copied, which makes parallel sampling and beam search cheap.
  10. Preemption, swap and recomputationWhen KV memory runs out under load, the scheduler pauses some sequences. It either swaps their blocks to host memory or drops them and recomputes the prefill later.
  11. Why attention is HBM-boundStandard attention writes an n by n score matrix to HBM. Bytes grow with n squared, so long sequences leave tensor cores idle.
  12. Online softmaxSoftmax needs a row max and a sum. Online softmax keeps running statistics so you can process the row in tiles and still get the exact result.
  13. FlashAttentionWhy standard attention is HBM-bound, how tiling plus online softmax computes exact attention in on-chip SRAM, and what FlashAttention-2/3 and decode variants change.
  14. FlashDecoding and split-KVDecode often has one query and a long KV cache. Split the KV across thread blocks, then reduce, so the GPU has parallel work again.
  15. FlashAttention-2 and 3Later FlashAttention kernels keep the same math. FA2 improves work partitioning. FA3 targets Hopper async copies and FP8.
  16. Quantization formats and scalesHow many bits a weight uses, and how scales stretch a coarse grid over a tensor. Per-tensor, per-channel and per-group scales trade overhead for quality.
  17. QuantizationINT8, FP8, INT4 and FP4. Fewer bits shrink weights, activations and KV. Match the format to memory-bound decode or compute-bound prefill, then eval quality.
  18. Weight-only vs activation quantizationW4A16 stores 4-bit weights and keeps 16-bit activations. W8A8 and FP8 quantize both sides so Tensor Cores run native low-precision matmuls.
  19. Activation outliersA few channels with huge magnitudes set a single INT8 scale and squash everyone else. LLM.int8() and SmoothQuant exist to stop that.
  20. GPTQ and AWQTwo post-training methods that compress a finished checkpoint to INT4 with a small calibration set. GPTQ uses second-order updates. AWQ protects salient channels.
  21. KV cache quantizationWeights are only half the story at long context. Quantizing KV to FP8 or INT8 or INT4 shrinks the cache, raising batch size or max context.
  22. Draft and verifyA cheap draft proposes k tokens. The target scores them in one forward. Rejection sampling keeps the target's distribution.
  23. Speculative decodingDraft-and-verify so one target forward accepts several tokens. Same output distribution. Real speedups when decode is memory-bound and acceptance stays high.
  24. Speculative acceptance rateSpeedup is expected accepted tokens divided by draft-plus-verify cost. Sweep k. Measure alpha on your traffic.
  25. Draft sourcesSmall LMs, prompt n-grams, Medusa heads, EAGLE and multi-token predictors. Pick for stack support and tokenizer match.
  26. Speculation versus batchingSpeculation is a latency tool for underfilled GPUs. At high batch it can steal compute from everyone in the continuous batch.
  27. Shared prompt prefixesSystem prompts, tools, few-shots and chat history repeat across requests. Those shared heads are the tokens prefix caching can skip.
  28. Prefix cachingReusing KV across requests that share prompts. Block hashing, RadixAttention, cache-aware routing, and why cached input tokens are billed cheaply.
  29. Automatic prefix cachingWith paged KV, each block is hashed from its tokens. Identical spans reuse the same physical blocks, so prefill can skip work without an application change.
  30. RadixAttentionSGLang stores retained KV in a radix tree keyed by token sequences. Shared prefixes are shared paths from the root, with LRU eviction at the leaves.
  31. Cache-aware routingPrefix cache lives on one replica. A load balancer that ignores cache state sprays requests and destroys hit rate. Send each request toward the replica that already holds its prefix.
  32. Multi-tier prefix cacheExtend a radix or block cache from GPU HBM to host DRAM and then SSD. Larger working sets, slower hits. Worth it for long-lived sessions, not short one-offs.

Scaling out

  1. Parallelism for inferenceTensor, pipeline, data and context parallelism: splitting a model across GPUs so weights and KV fit, and single-request latency stays under SLO.
  2. Tensor parallelismSplit each layer across GPUs so every token uses every rank. Column-parallel and row-parallel matmuls plus all-reduces do the work.
  3. Pipeline parallelismAssign blocks of layers to different GPUs or nodes. Inter-stage traffic is small, but decode leaves pipeline bubbles that waste GPU time.
  4. Data parallelism for inferenceIndependent replicas of an already-sharded model. Each replica has its own KV pool. Throughput scales; single-request latency does not.
  5. Total vs active parametersA mixture-of-experts model stores far more weights than it uses for each token. Memory follows the total. Compute follows the active count.
  6. Serving mixture-of-experts modelsExpert parallelism, all-to-all dispatch/combine, and load balancing for sparse MoEs like DeepSeek-V3 — where memory tracks total params but FLOPs track active ones.
  7. Expert parallelism and all-to-allExpert parallelism puts different experts on different GPUs. Each MoE layer sends tokens to their experts and back with two all-to-all steps.
  8. Hot experts and load imbalanceRouters rarely spread tokens evenly. One hot expert can overload its GPU, and because the combine step waits for everyone, the whole layer slows down.
  9. Different EP widths for prefill and decodePrefill and decode send very different numbers of tokens to each expert, so the best expert-parallel width is usually different for each phase.
  10. Prefill–decode interferenceA long prefill on the same GPU as decode steals SMs and HBM bandwidth, so everyone else's TPOT spikes.
  11. KV transferDisaggregation ships the KV cache to a decode worker. Multi-GB cold prompts on slow links blow the TTFT budget.
  12. Disaggregated prefill and decodeRunning prefill and decode on separate GPU pools so each phase gets its own parallelism, batching, and hardware — and when colocated chunked prefill is enough.
  13. Sizing prefill and decode poolsSet GPU counts from prefill GPU-seconds and decode GPU-seconds per request, not a 50/50 split.
  14. Disaggregation failure modesOrphan KV, slow-start decode, lost cache affinity and version skew are the outages you do not have when pools are colocated.
  15. Why long context is expensivePrefill attention grows with n squared. KV and decode reads grow with n. A million-token prompt is not eight times a 128k prompt.
  16. Context parallelismShard the sequence across GPUs and pass KV or Q around a ring. This cuts long prefill latency. On decode it often hurts.
  17. Serving long contextsMillion-token prompts: quadratic prefill attention, linear KV growth, context parallelism, offloading, sparse/hybrid architectures, and how providers price the pain.
  18. Sliding-window and hybrid attentionLocal windows cap KV growth. Hybrids mix local layers with periodic global attention. Prefer them when the product does not need lossless full attention.

Serving systems

  1. Anatomy of a serving engineAPI, scheduler, KV manager, executor, sampler, detokenizer. One forward pass packs every sequence the scheduler admitted.
  2. Serving engines: vLLM, SGLang and TensorRT-LLMHow modern engines are built, how they differ, and which knobs move TTFT, TPOT and tokens per GPU-second.
  3. vLLMThe broad default engine. PagedAttention, a unified iteration-level scheduler, and wide model coverage.
  4. SGLangBuilt for prefix-heavy and structured workloads. RadixAttention shares KV the way a filesystem shares path prefixes.
  5. TensorRT-LLMCompile a model into a GPU-specific engine. You pay a rebuild cost and gain tightly tuned NVIDIA kernels.
  6. Engine knobs that matterTensor parallel size, max model length, GPU memory utilization, quantization, speculation and prefix caching. Ignore the long flag list until these are right.
  7. Goodput and realistic workloadsCapacity is throughput that still meets the SLO, measured on your length mix, not on 512/128 prompts.
  8. Closed-loop vs open-loop loadFixed concurrency finds the latency–throughput curve. An arrival process shows what happens when QPS exceeds goodput.
  9. Benchmarking inferenceMeasuring honestly: workloads, load generators, concurrency sweeps and the mistakes that make numbers meaningless.
  10. Concurrency sweepsWarm up, step concurrency, plot latency against throughput, and take the highest point that still meets the SLO.
  11. Benchmark pitfallsCache hits, guessed tokens, client bottlenecks and peak-only slides are how inference numbers become marketing.
  12. Why round robin fails for LLMsTwo HTTP requests can differ by 100× in GPU-seconds. Equal request counts do not equal equal load when cost, cache and streams are uneven.
  13. Replica scoringBeat round robin with an explicit score from prefix affinity, queue depth, KV headroom and adapter locality. Hard filters first, then pick the max.
  14. Routing, load balancing and schedulingLLM-aware gateways: KV-cache-aware routing, token-based rate limits, priorities and fairness.
  15. Token-based rate limits and gateway policyLimit estimated tokens, not request counts. Add priorities, fair queues, cancellation and hard max_tokens at the gateway.
  16. Admission control and load sheddingWhen offered load exceeds goodput, queue with a deadline, shed batch first, then reject with Retry-After. Do not OOM the replica.
  17. LoRA adaptersLoRA fine-tunes a model by training a small add-on beside frozen weights. The add-on is tens of megabytes, so one base model can carry thousands of them.
  18. Multi-LoRA and multi-model servingServing many fine-tuned adapters and models on shared GPUs efficiently.
  19. Multi-LoRA batching and adapter pagingOne base model can serve thousands of LoRA adapters if one forward pass handles requests for different adapters, and cold adapters are paged in from host memory.
  20. GPU sharing with MIG, MPS and time-slicingMIG, MPS and time-slicing let several workloads share one GPU. Each trades isolation against density, and each fits different workloads.
  21. Tenant isolation and noisy neighborsWhen tenants share GPUs, one heavy customer can push everyone else's latency up. Admission caps, fair scheduling, priority classes, separate pools and per-request limits keep tenants apart.

Production and economics

  1. Scaling signals for LLM serversScale on waiting queue, KV cache and TTFT, not on CPU or GPU utilization.
  2. Inference cold startsA new replica is not ready when the pod starts: image, weights and CUDA graphs take minutes.
  3. Autoscaling and GPU fleet managementScale GPU replicas on queue depth, KV pressure and SLOs — not GPU util — then plan fleets around cold starts, warm pools and heterogeneous hardware.
  4. Scale-to-zero for inferenceScale-to-zero saves GPU-hours on sparse models, and turns the first request into a multi-minute stall.
  5. GPU fleet planningHold a reserved baseline, burst on demand, and put each model on the GPU class that can meet the SLO at the lowest cost.
  6. Replica drain and GPU failuresStop routing new requests before you kill a replica, and treat GPU and NVLink loss as a first-class outage.
  7. The token cost formula$/M tokens is pool dollars per hour divided by sustained tokens per second. Utilization is the silent multiplier.
  8. Input vs output token costPrefill, cached prefixes and decode have different cost drivers, so one "tokens" counter hides where to optimize.
  9. The economics of inferenceTurn $/GPU-hour into $/million tokens with utilization and throughput, then decide API vs self-host and price products with real margins.
  10. API vs self-hostSelf-hosting wins when GPUs stay busy on a model you accept. APIs win when volume is spiky or frontier quality matters more than cents.
  11. Pricing inference productsPrice from p95 user cost, not the mean, and do not promise cheap tokens and high tokens-per-user on the same GPU.
  12. On-device decode ceilingsDecode speed on a laptop or phone is roughly bandwidth divided by weight bytes, and client silicon is far below an H100.
  13. Edge runtimes and GGUFllama.cpp and GGUF for cross-platform desktop, MLX or Core ML for Apple, ExecuTorch and ONNX for mobile graphs, WebGPU for zero-install previews.
  14. On-device and edge inferenceRun models on laptops, phones and browsers when latency, privacy or offline matter — with napkin math for bandwidth-bound decode and the right runtime.
  15. Small models and hybrid routingRun a tiny on-device model for easy or private work, and escalate hard queries to the cloud with consent.
  16. Edge KV, thermals and downloadsCap context, budget heat and battery, and never hide a multi-GB download inside the app binary.
  17. Serving embeddings and rerankersEncoder-only models do one forward, no growing KV. Batch by sequence-length buckets. Indexing is a throughput job. Query embed is a latency job.
  18. Serving embeddings, rerankers, speech and diffusionInference beyond chat LLMs. Encoders, speech, diffusion and classic ML each have a different bottleneck and a different batcher.
  19. Serving speech modelsASR cost scales with audio duration. Batch similar clip lengths. Streaming TTS is judged on time-to-first-audio, not realtime factor alone.
  20. Serving diffusion modelsLatency is steps times step time. Distillation beats micro-optimizing a 50-step loop. Interactive image gen is often batch 1.
  21. Serving classic ML modelsGBDTs and small nets belong on CPU ONNX or Triton FIL. A GPU would sit idle. Latency budgets are often sub-5 ms p99.
  22. Capacity from goodputConvert peak RPM to tokens per second, then divide by SLO-constrained goodput per replica. Peak blog tok/s is the wrong denominator.
  23. Case study: designing an LLM serving platformA 70B-class model and LoRA fleet at 50k RPM under TTFT and TPOT SLOs. Capacity math, pools, gateway and failure drills.
  24. Interactive and batch poolsRun chat on reserved replicas with TTFT SLOs. Put offline jobs on spare or spot capacity with preemption so they cannot steal the interactive KV.
  25. Platform failure drillsPlan for a dead GPU node, a traffic spike faster than boot, and a bad checkpoint. Canary, N+ capacity and an explicit overload policy.

Engineering craft and career

The skills that get you from junior to SDE2 and on track for SDE3: craft, ownership, communication and interviewing.

Levels and expectations

  1. The four leveling axesScope, autonomy, technical complexity and influence — the four questions every engineering ladder is really asking.
  2. What the levels meanJunior, SDE2, SDE3 and staff: scope, autonomy, impact and influence at each level.
  3. Junior and SDE2Juniors finish well-scoped work cleanly. SDE2s own features end to end without someone breaking the work into steps.
  4. Senior and staffSenior owns ambiguous multi-quarter problems and multiplies the team. Staff is a different job shape, not a louder senior.
  5. Mapping titles across companiesSenior at a startup may be mid-level at big tech. Map with scope, autonomy, complexity and influence — not vanity strings.
  6. Leveling self-assessmentPick a public ladder, score three recent pieces of work on the four axes, and mark what you show consistently versus once.
  7. The first 90 daysAgree 30/60/90 outcomes with your manager so ramp is visible, not a vibe.
  8. Reading an unfamiliar codebaseTrace one request from an entry point. Tests and git history beat reading src/ top to bottom.
  9. Asking good questionsShow what you tried, time-box being stuck, and batch questions so help is cheap to give.
  10. Ship something small in week oneA tiny production change teaches the real workflow. A tutorial app on your laptop teaches none of it.
  11. Ramping up on a new team and codebaseYour first 90 days: read code by tracing paths, ask well, ship early, and map people as well as systems.
  12. Closing loopsStay with the outcome, not the ticket. Every commitment ends done, cancelled, or deferred with an owner and a date.
  13. Reliability as reputationDo what you said, by when you said, or renegotiate before the deadline. Quiet misses train people to discount you.
  14. Done means verifiedTests, monitoring, a rollback path, and a check in production are part of done — not polish if there is time.
  15. Proactive ownershipFind problems before they are tickets, package them in risk language, and make yourself replaceable on purpose.
  16. OwnershipActing like an owner: close loops, care about outcomes, and be the person things do not fall through with.

Craft of code

  1. Software complexityComplexity shows up as change amplification, cognitive load and unknown unknowns. The next stranger's change is the cost that matters.
  2. Deep modulesA deep module hides a lot of complexity behind a small interface. Shallow pass-through layers add names without adding leverage.
  3. Writing code others can maintainManaging complexity with deep modules, clear naming, honest comments, consistent style, and error strategies that shrink the blast radius of change.
  4. Naming and commentsNames encode role and units. Functions do one thing at one level. Comments explain why, not what.
  5. Error handling and illegal statesFail fast, make errors explicit, and redesign so some failures cannot happen. Swallowing errors turns production into archaeology.
  6. Duplication vs the wrong abstractionDRY is about knowledge, not text. Prefer duplication over a shared helper you cannot name. YAGNI for imaginary second use cases.
  7. Interfaces that age wellDesign the interface for the caller's job. Take the smallest inputs, make illegal states hard, and keep one obvious way to do each thing.
  8. Test levelsUnit, integration, contract, and end-to-end tests buy different confidence per minute. Pick with cost, not habit.
  9. Test doublesA stub returns canned data. A fake is a small working stand-in. A mock also checks how it was called. Prefer fakes and real dependencies.
  10. Testing behaviorAssert inputs and outputs the next engineer cares about. Do not lock tests to private call graphs, logs, or getters.
  11. Testing strategyWhat to test, at which level, and how to keep a suite fast, reliable, and worth the maintenance — from pyramid trade-offs to production safety nets.
  12. Flaky test policyA flake fails without a code change and trains the team to ignore red CI. Quarantine it the same day; fix or delete it within days.
  13. Testing in productionPre-prod tests never see every config, data shape, or traffic pattern. Flags, canaries, and synthetics catch what the suite cannot invent.
  14. Reviewable pull requestsSmall diffs, a stranger-readable description, and a self-review before you ask anyone else.
  15. What to look at in a reviewDesign, then correctness, then tests, then readability, then nits. LGTM means you would own the merge.
  16. Review toneQuestions and labeled nits beat commands. Explain why. Specific praise teaches what to repeat.
  17. Code reviewAuthor small, reviewable PRs and give reviews that improve both the code and the author — design first, tone that teaches, turnaround that unblocks.
  18. Review turnaroundSlow review is a delivery bug. First response within a business day; silence is worse than a redirect.
  19. Block vs approveBlock what is wrong, unsafe or expensive to undo. Approve improvements and track leftover polish.
  20. Ops and security in reviewAuthz, secrets, PII, migrations, flags and load. You do not need to be a security engineer to ask who can call this.
  21. The scientific debugging loopObserve a concrete failure, form a killable hypothesis, change one variable, narrow, repeat. Write it down.
  22. Reproduce and minimizeThe smallest steps, inputs and environment that make the bug happen reliably. Without a repro you are guessing.
  23. Bisecting bugsBinary search the search space — code path, commits, input, config, cluster. git bisect is log2(N) builds.
  24. DebuggingA scientific loop for finding bugs fast — reproduce, hypothesize, bisect, instrument — from local failures to production incidents without making things worse.
  25. Reading debug evidenceStacks, logs, metrics and traces. The exception escaped somewhere; the mistake may be a frame earlier.
  26. Production debuggingCorrelate, stabilize, capture evidence. Do not thrash. Separate mitigation from the root-cause fix.
  27. Closing the debug loopA fix without a regression test is temporary. Write what you saw, what was true, and how it cannot silently return.
  28. Characterization testsPin what legacy code does today before you clean it, so a refactor cannot drift in silence.
  29. Seams and dependency breakingA seam is a place you can change behavior without editing the hard spot. Create one before you rewrite the algorithm.
  30. Tiny refactor stepsOne structural change, tests green, commit. Then change behavior in a different PR.
  31. Refactoring and legacy codeChange code you did not write without breaking it. Start here for characterization, seams, tiny steps, and incremental migrations.
  32. Incremental migrationsReplace a system the way a strangler fig replaces a host — route by route, with expand/contract and parallel run, never a six-month freeze.
  33. Selling tech debtFund cleanup with incident rate, speed, and risk — not "this code is messy" — and show progress on a dashboard.
  34. AI coding modesAutocomplete completes patterns you already understand. Chat drafts and explains. Agents edit many files against a pass/fail signal.
  35. Agent verify loopsAn agent only helps when it can fail a test, a typecheck, or a repro script. You define done. You still review the merge.
  36. Working effectively with AI coding toolsUse autocomplete, chat, and coding agents to go faster without shipping code you do not understand — context, verify loops, and protecting your skill growth.
  37. Prompting AI coding toolsLoad a goal, an example, constraints, and non-goals. Point at the files that matter. Keep project instruction files short and true.
  38. Reviewing AI-generated codeReview AI output with the rigor of a new-hire PR. Characteristic failures are invented APIs, skipped auth, and tests that assert mocks.
  39. Learning with AI toolsSeniors multiply with AI. Juniors risk never building the mental models that make them SDE2. Protect growth on purpose.

Owning projects

  1. When to write a design docSkip the doc for small reversible work. Write one when the blast radius is large, the approach is unclear, or you need buy-in.
  2. Design docs and RFCsWriting design docs that get to a good decision quickly — structure, alternatives, review, and when a one-pager is enough.
  3. Design doc templateContext, goals and non-goals, the proposed design, real alternatives, then risks and rollout. Cut empty sections.
  4. Writing design docs for reviewersLead with the decision on the first screen. Support it with diagrams and numbers. Ask for scrutiny, not applause.
  5. Running a design reviewPick reviewers on purpose, go async first, meet only for stuck threads, and record the decision so code review does not re-litigate it.
  6. Estimate vs commitmentAn estimate is a range under uncertainty. A commitment is a date you agree to hit, usually by cutting scope. Do not give one when asked for the other.
  7. Work breakdown and spikesSplit work until each leaf is about one or two focused days. Time-box unknowns as spikes before you pretend to know their cost.
  8. Estimation and planningBreaking work into small tasks, estimating with ranges, de-risking early, and communicating slips before they become surprises.
  9. Estimate ranges and paddingQuote a range and a confidence, then multiply the raw sum out loud for unfamiliarity. Hidden padding is how trust dies.
  10. De-risking and milestonesPull the scary integration forward, then ship user-value slices. Easy UI first feels like progress and hides the landmine.
  11. Evidence-based schedulingTrack estimate versus actual on small tasks, then multiply new work by your personal factor instead of summing hopeful guesses.
  12. Communicating slipsSay the assumptions with the number. When they break, renegotiate early with options — cut scope, move the date, or (rarely) add people.
  13. DRI and kickoffOne named owner, a short kickoff, and a definition of shipped that is more than code on main.
  14. Green, yellow, red statusStatus is a forecast of launch, not a diary of tasks. Yellow on Tuesday beats red the day before.
  15. Dependencies and unblockingOther teams will not feel your deadline. Ticket, follow up, escalate with options — and parallelize while you wait.
  16. Launch readinessMerged to main is not launched. Walk monitoring, rollout, rollback, docs, and a named owner before the flag goes to 100%.
  17. Driving projects to launchKickoffs, status, dependencies, unblocking, launch checklists, and a finish decision-makers recognize as shipped.
  18. Discovery and deliveryDiscovery figures out what to build. Delivery builds and ships it. Engineers belong in both, as peers of PM and design — not a ticket factory.
  19. Cheaper product cutsStart from the user problem and success metric, then offer an MVP with a cost and a risk — not a yes or a no.
  20. Working with product managers and designersProduct sense for engineers — shaping scope, trade-off conversations, feasibility spikes, and building the right thing with PMs and designers.
  21. Feasibility spikesSurface a hard constraint before design is locked. Time-box a spike and come back with a recommendation and residual risk.
  22. Working with designersDesigners are partners on empty, error, and permission states — not pixel people you implement after the fact.
  23. Product-minded habitsCuriosity about the user, interest in metrics, and ownership after launch — not a PM title.
  24. Shipping with observabilityA feature is not done when the happy path works locally. Done means someone half-asleep can tell whether it is healthy.
  25. Handling a pageAcknowledge, assess blast radius, mitigate first, escalate when needed, communicate. Diagnose after users are safe.
  26. Runbooks and toilWrite the checklist you wish existed at 2 a.m., then spend the next days making that page never happen the same way.
  27. Owning your code in productionOn-call, alerts, runbooks, and the loop that makes you trustworthy — and better at building systems that survive users.
  28. Production postmortemsBlameless writeups so the system learns. Your job is timeline facts, contributing factors, and action items you actually finish.

Working with people

  1. Bottom line up frontPut the decision, ask or outcome in the first sentence. Context comes after.
  2. Writing for skimmersAssume a tired reader on a phone. Headings that say something, one idea per paragraph, links instead of dumps.
  3. Status and decision updatesShared templates for status, incidents and decision requests so readers know where to look.
  4. Async etiquetteChat is a shared inbox. Put the full ask in message one, use threads, and match the medium to the urgency.
  5. Written and verbal communicationLead with the point, write for skimmers, and adjust depth for engineers, PMs and executives.
  6. Audience altitudeSame facts, different height. Mechanism for engineers, impact for PMs, decision and ask for executives.
  7. Meetings and demosOpen with the purpose, show the user path, and leave with a decision, an owner and a date.
  8. Building trust before you need itTrust is credit you earn in quiet weeks. Keep small commitments, share context early, make others look accurate.
  9. Disagreeing wellCritique the proposal, not the person. Bring facts, options and a recommendation, then disagree-and-commit.
  10. Influence without authorityShrink the ask, do their first draft, name shared upside, and escalate with options rather than a complaint.
  11. Collaboration and disagreementBuild trust before you need it, disagree with data and options, and make cross-team asks easy to grant.
  12. Cross-team asksOther teams have roadmaps and review queues. Make saying yes the path of least resistance.
  13. Glue workThe connective tissue that lands projects — and stalls careers if it leaves no artifact. Do it, name it, time-box it.
  14. Psychological safetyPeople can ask a dumb question, admit a miss, or challenge a plan without being punished. That is not low standards.
  15. Mentors, coaches and sponsorsMentors advise, coaches question, sponsors put reputation on the line. You need a mix, not one perfect person.
  16. Mentorship: getting it and giving itDistinguish mentors, coaches and sponsors — then ask well, mentor juniors and grow others as an SDE2.
  17. Asking for mentorshipAsk with a bounded cadence, an agenda and a specific question tied to an artifact. Vague "be my mentor" contracts get declined.
  18. Mentoring juniorsOnboard with a first-week win, teach through reviews, and coach with hints unless the cost of delay is a sev.
  19. Building a mentor benchSeveral partial relationships beat one oversubscribed senior. Split craft, career, domain and peer. Cap how many people you mentor deeply.
  20. Sponsorship and growing othersPromotion committees look for people you raised, not Slack answers. Track mentee outcomes the way you track projects.
  21. Running your 1:1sA 1:1 is not a status meeting. Bring an agenda, rotate themes, and write down the decisions.
  22. Asking and receiving feedbackSpecific prompts soon after the work beat "any feedback?" Receive by summarizing first. Evaluate truth later.
  23. Giving feedback with SBISituation, Behavior, Impact. Observable events, not character. Praise in public when it will not embarrass; correct in private.
  24. Brag documentsA living note of what you did, the impact, links, and kudos. Review cycle you then compile, not invent.
  25. One-on-ones, feedback and performance reviewsRun your 1:1s, ask for and act on feedback, and keep a brag document through review cycles.
  26. Performance review cyclesSelf-review, peers, calibration. Make the true story easy to find, and never let a serious concern appear for the first time on the form.

Growing your career

  1. The scope ladderTitles track the size of problem you can be trusted with — task, feature, project, then a multi-team problem.
  2. High-leverage workImpact per hour hides in on-call themes, stuck metrics, roadmap gaps, and daily team friction — not in tickets labeled "promo project."
  3. Growing your scopeFrom tasks to features to projects to problems: finding and taking on higher-leverage work.
  4. Volunteering for scopeFinish before you expand. Volunteer for the gap your next level requires, with an explicit metric, time box, and non-goals.
  5. Making impact visibleCalibration cannot see work you never wrote down, measured, or demoed. Leave artifacts on purpose.
  6. Promotion is lagging recognitionYou get the title after you have already been operating at the next level, usually for more than one cycle.
  7. Building a promo packetMap two or three linked examples to each next-level criterion. Cut anything that only proves your current level.
  8. Working with your manager on promoMake the goal explicit, share the brag doc early, and ask for scope that fills gaps — not only adjectives.
  9. How promotion worksEvidence, packets, calibration and the manager conversations that turn next-level work into a title.
  10. Promo blockers and when to moveMost stuck cases are missing scope, invisible impact, or a team with no next-level problems. Sometimes leaving is the plan.
  11. Engineering leverageImpact divided by time. Ask what the next hour is worth, not whether you looked busy.
  12. Protecting focus timeHard engineering needs uninterrupted stretches. Treat focus blocks like meetings with a critical stakeholder — you.
  13. Tooling and automationTime spent learning your editor, shell, and a boring automation threshold compounds for years.
  14. Saying noEvery yes is a no to something else. Seniors allocate help on purpose, with timed yes, partial yes, or an explicit tradeoff.
  15. Productivity, focus and sustainabilityLeverage, deep work, saying no, and a pace you can still think clearly at in year five.
  16. Sustainable paceBurnout is a prolonged mismatch between demand and recovery. Treat the warning signs like a production alert.
  17. Company stageBig tech, growth-stage and early startup trade learning, scope, pay and risk. Pick the trade for this season.
  18. Choosing teamsOver a two-year window, manager plus business importance plus technical growth usually beats the company logo.
  19. T-shaped skillsDepth in one stem the market pays for, plus a crossbar of literacy so you can collaborate and switch.
  20. Career strategyChoosing companies, teams and specialties deliberately: big tech vs startups, generalist vs specialist.
  21. Evaluating startup offersSeparate cash, equity, runway and scope. A large option grant is a probability distribution, not a salary.
  22. When to switch jobsSwitch when growth is structurally blocked. Stay when a scope jump or rare exposure is coming. Leave with reputation intact.
  23. A light public profileDiscoverability and proof, not an influencer side hustle. Short posts, one talk, small PRs, factual LinkedIn.
  24. Owning ambiguitySenior work starts when the problem statement is incomplete. You write down the goals, constraints, and success metrics for others.
  25. Technical leadership without managementYou do not need reports to lead. Use design reviews, standards, short strategy, and reviews that teach — and keep coding enough to stay credible.
  26. The path to SDE3 and beyondWhat senior engineers do differently: technical leadership, multiplying others and owning ambiguity.
  27. Navigating organizationsTechnical excellence that ignores organizations dies in a ditch. Alignment is getting the right people to agree, then keeping that agreement alive.
  28. Staff archetypesTech lead, architect, solver, right hand — and the IC vs manager fork. Pick a game on purpose. Do not become a manager because promo is blocked.
  29. A plan from SDE2 to SDE3A 12–24 month template with explicit gaps, one senior-shaped project, wider influence, and anti-patterns that feel like progress.

Interviewing

  1. How interview loops workRecruiter screen, coding, onsite, debrief, level, team match, offer. Each round is an independent hire/no-hire that a committee stacks.
  2. What interview rounds measureCoding scores process under time. Design scores decomposition and trade-offs. Behavioral scores ownership. The debrief scores your level from all three.
  3. Interview preparation strategyHow loops work, what each round measures, and a 6–10 week prep plan that keeps coding, system design and behavioral on track.
  4. Interview prep timelineA 6–10 week plan with overlapping tracks — coding first, design overlapping, behavioral from day one, mocks last.
  5. Mock interviewsMocks and recordings fix communication failures that silent grinding never surfaces. Grade process, not vibes.
  6. Batching applicationsApply in a 2–3 week window so offers overlap. Do not open the pipeline before you can clear a timed coding screen.
  7. Startup vs big-tech interviewsBig tech wants patterns, design frameworks and STAR stories. Startups want practical coding, take-homes and product conversation.
  8. The interview problem-solving processClarify, work examples, state a brute force, then optimize, code, test and analyze. Interviewers grade the path.
  9. Interview patternsLearn the trigger, not a code dump. Two pointers, windows, hashing, graphs and a few more cover most mediums.
  10. Communicating while you codeNarrate intent, not keystrokes. Use hints, pick one language, and recover from bugs out loud.
  11. Coding interviewsPatterns, problem-solving process and communication for algorithm interviews — plus how practical coding rounds differ.
  12. Practical coding roundsSome loops ask you to build, debug or review in a toy repo. Clarity, tests and trade-off talk beat clever asymptotics.
  13. How to practice coding interviewsCurated lists and spaced re-solves beat random volume. Readiness is an unseen medium in about 25 minutes.
  14. Coding interview mistakesCoding immediately, going silent, and waiting for the optimal solution are the usual fails. Name the miss and recover.
  15. What behavioral rounds assessOwnership, conflict, ambiguity, impact, mentorship and learning. Past behavior is the predictor they use.
  16. STAR answersSituation, Task, Action, Result, plus Learning. Spend half the time on what you did.
  17. Building a story bankEight to ten deep stories, each mapped to two or three themes. Rehearse out loud. Update after launches.
  18. Behavioral interviewsStory banks, STAR(L), and signaling the level you want — ownership, conflict, ambiguity and impact with numbers.
  19. Signaling level with storiesThe same incident can sound junior or senior. Emphasize scope, autonomy and influence for the level you want.
  20. Failure and conflict storiesHonest miss, bounded blast radius, systemic fix. Disagreement with data, then commit. Never trash people.
  21. Handling behavioral deep divesInterruptions are interest. Prepare alternatives, one metric, one miss inside the success, and who else was involved.
  22. Compensation componentsCompare total compensation across base, bonus, sign-on and equity with real vest math — not base alone.
  23. Level vs compensationAt leveled companies, jumping a level usually moves TC more than squeezing the top of a lower band.
  24. Offers and negotiationComp components, why level beats base, market data, negotiation scripts and evaluating startup equity without wishful math.
  25. Negotiating an offerDo not name the first number. Get the offer in writing, then ask with enthusiasm and a specific, market-grounded target.
  26. Evaluating startup equityTranslate percentages into scenarios. Price startup equity as a risk asset, not a promised TC line on levels.fyi.
  27. Juggling multiple offersSpreadsheet year 1 and years 2–4, score non-comp, then tell your first choice what would make accepting easy. Use deadlines honestly.
  28. Accepting and declining offersConfirm the package in writing, withdraw other processes promptly, and decline briefly with the door open. Reneging damages reputations.

Product

Product thinking for engineers: finding problems worth solving, measuring what matters, experimenting, growing and pricing.

Product thinking and discovery

  1. Outcomes vs outputShipping is output. Outcomes are changes in user or business behavior. The build trap is optimizing the first and ignoring the second.
  2. Four product risksEvery failed product fails at least one of value, usability, feasibility or viability. Name the biggest risk before you write code.
  3. Product thinking for engineersOutcomes over output — why engineers who understand users and business build better software. Start here for the map.
  4. The product trioA PM, a designer and an engineer share outcomes. Discovery is shared work, not a handoff, and product management is not project management.
  5. Product-minded engineersAsk what problem, for whom and how you will know — then reshape the solution, cut scope to keep the outcome and watch the numbers after ship.
  6. Problem vs solutionFall in love with the struggle, not the dashboard. Solutions are disposable; the pain is not.
  7. Finding problems worth solvingPain intensity, frequency and willingness to pay — separating hair-on-fire problems from nice-to-haves.
  8. Scoring problemsIntensity, frequency, workarounds and willingness to pay — the filter between hair-on-fire and nice-to-have.
  9. Where problems come fromMine support, sales, analytics, dogfooding and observation. Cross-check before a request becomes strategy.
  10. Problem statementsWho, the job, the obstacle, the cost, the workaround, the evidence — then a hypothesis that can fail.
  11. What is a job to be donePeople hire a product to make progress in a circumstance — not to buy a feature list or a demographic label.
  12. Jobs to be doneWhat customers hire a product to do — and how JTBD reframes competition and features.
  13. Forces of progressPush and pull move people toward a new product. Anxiety and habit hold them. Great products fail when the last two win.
  14. Switch interviewsReconstruct the last time someone hired or fired a product. Past timelines beat hypothetical "would you switch?"
  15. Job storiesWhen circumstance, I want progress, so I can outcome. A usable job statement you can design and cut scope against.
  16. The Mom TestTalk about their life, not your idea. Ask about the past, not the future. Compliments are false positives; past behavior and money already spent are not.
  17. User interviewsRecruit the segment in your problem statement, run a 30-minute script that stays in their world and take notes in their words. Five to eight good interviews beat fifty mediocre surveys.
  18. User researchInterviews that avoid false positives, usability tests with five users, and synthesizing findings. Start here for the map.
  19. Usability testingGive realistic tasks, ask people to think aloud and do not help until they are stuck. About five users find most issues in a given design; then fix and retest.
  20. Research synthesisAffinity-map quotes into observations, insights and ideas — in separate columns. An insight is surprising, grounded in multiple data points and actionable.
  21. Opportunity solution treesA living map from outcome to opportunities to solutions to tests — so you compare ideas under the same customer need.
  22. Continuous discovery and assumption testingWeekly customer contact, opportunity solution trees and cheap tests before building.
  23. Riskiest assumptionsState what must be true for an idea to work, then test high-hurt and high-uncertainty beliefs first.
  24. Cheap discovery testsFake doors, concierge, Wizard of Oz and prototypes — spend days, not months, to see if an assumption holds.
  25. A week of discoveryOne outcome, two interviews, one cheap test, a kill criterion and a Friday decision — no production feature required.

Strategy and planning

  1. Vision vs strategy vs roadmapVision is the destination. Strategy is a focused bet under constraint. Roadmaps and tactics execute. Do not mix the four layers.
  2. Rumelt's strategy kernelA good strategy has a diagnosis, a guiding policy and coherent actions. Fluff, goal-lists and twelve "priorities" are not strategy.
  3. Product strategy and positioningVision, strategy, positioning and differentiation: deciding where to play and how to win. Start here for the map.
  4. Product positioningPosition from the alternative customers already use, not from the category you wish you were in. Dunford's five components, in order.
  5. The strategy one-pagerIf the strategy does not fit on one page, it is not focused enough to execute. Include non-goals and how you will know.
  6. Opportunity costThe real price of a yes is the best alternative you did not ship.
  7. RICE scoringReach × Impact × Confidence ÷ Effort — a comparable score that structures debate, not a verdict.
  8. PrioritizationRICE, ICE, Kano, cost of delay and opportunity cost: choosing what not to build.
  9. ICE, Kano and cost of delayFaster scoring when reach is similar, categories of need, and how much waiting costs per week.
  10. Capacity allocationReserve capacity for growth, commitments, reliability and debt so incidents are not the only planning process.
  11. Cheaper versionsEngineers win prioritization with honest effort, thinner slices, and a written reason when they override a score.
  12. Writing a PRDA short doc that answers problem, users, metrics, non-goals and P0 behavior — sized to the bet.
  13. Requirements vs specsProduct owns observable behavior. Engineering owns how it is built. Mixing them too early locks the wrong decisions.
  14. PRDs, PR/FAQs and specsWriting product requirement docs and Amazon-style PR/FAQs that align teams.
  15. PR/FAQ and working backwardsWrite the press release as if the product shipped. If you cannot make a customer care, kill the idea cheaply.
  16. Acceptance criteriaTurn P0 requirements into testable conditions, including the edge cases that bite in production.
  17. Living PRDsRight-size the doc, mark status, publish a changelog, and pair it with an eng design doc for the how.
  18. Outcome roadmapsA roadmap tells the organization which problems you will solve, in what rough order. It is not a Gantt contract of seventeen features by March 31.
  19. Now / next / laterThree columns that communicate sequence and uncertainty without fake dates. Outcomes on top, bets underneath, Later without a calendar.
  20. Roadmaps and OKRsOutcome-based roadmaps, now/next/later, and OKRs that guide rather than constrain. Start here for the map.
  21. Writing OKRsObjectives are qualitative aims. Key results are measurable evidence — typically 2–4 per objective. Ship lists, sandbagging and fake cascades are not OKRs.
  22. Roadmap capacityPartition capacity so sales commitments and discovery coexist without lying. Plan the quarter from strategy plus eng-weeks, and leave slack.
  23. Riskiest assumptionAn MVP tests the belief that, if false, kills the idea — written so it can fail.
  24. Types of MVPConcierge, Wizard of Oz, single-feature wedge, or glued tools — pick the cheapest honest signal.
  25. MVPs and scopingBuilding the smallest thing that tests the riskiest assumption, and iterating.
  26. Appetite and shapingFix the time, vary the scope: how much is this idea worth, not how long will it take.
  27. Cutting scopeCut personas, surfaces and polish first. Never cut the aha, trust basics, or instrumentation.
  28. Pivot, persevere, or killPre-declare the signals that mean continue, change the job, or stop — a clean kill is a successful MVP.

Metrics and experimentation

  1. The north star metricOne number that captures customer value, tends to lead revenue and that teams can actually move. Write it as a count of actors doing a valuable action.
  2. Metric treesBreak the north star into 3–5 input metrics teams own. When the star is flat, diagnose which factor stalled instead of shipping random features.
  3. Product metrics and the north starChoosing a north star that reflects customer value, decomposing it into input metrics teams own, and pairing it with guardrails. Start here for the map.
  4. Guardrails and vanity metricsPair every primary metric with 2–4 guardrails so you cannot silently destroy trust. Vanity metrics move easily and do not change what you build next week.
  5. AARRR and leading metricsPirate metrics map the journey. They do not replace a north star. Prefer metrics that move in weeks over ones that only confirm last quarter.
  6. Event tracking plansName events as object_action, merge anonymous identity in the first session and treat the tracking plan as a contract. Bad events in a fancy UI stay bad.
  7. Product funnelsA funnel is an ordered sequence of events for the same actor. Conversion multiplies across steps, so size each lever with arithmetic before debating features.
  8. Product analyticsDesigning event tracking plans, reading funnels and cohorts, writing the SQL that answers product questions, and avoiding mix-shift illusions. Start here for the map.
  9. Simpson's paradox in mix shiftEvery segment can get worse while the overall rate improves, because more traffic came from the high-converting segment. Segment before you celebrate.
  10. Analytics SQL and debuggingA few warehouse shapes — DAU, windowed funnels, cohort retention — answer most product questions. When a number looks wrong, check definition, deploy, identity and mix before rewriting the query.
  11. Randomization and causalityWhy a dashboard cannot tell you what caused a change, and how random assignment isolates the effect of a product tweak.
  12. Designing an experimentWrite the hypothesis, primary metric, guardrails, unit and exposure definition before anyone looks at a p-value.
  13. A/B testing and experimentationDesigning randomized experiments, sizing samples with real arithmetic, reading lift and confidence intervals, and shipping only what the data actually supports.
  14. Sample size for A/B testsHow many users you need, why tiny lifts demand huge traffic, and how to pick an MDE you would actually ship on.
  15. Reading A/B test resultsLift, confidence intervals and p-values, and why a significant result can still be the wrong ship.
  16. Peeking at experimentsChecking the p-value daily and shipping when it dips below 0.05 inflates false positives from 5% into the teens or worse.
  17. Metric fishingTest 20 metrics and something will be "significant." Ship on one pre-registered primary, and treat the rest as leads.
  18. Experimentation pitfallsPeeking, metric fishing, sample ratio mismatch, novelty effects, network interference, CUPED and Twyman's law — how trustworthy experiments go wrong and how to catch them.
  19. Sample ratio mismatchIf you asked for 50/50 and observe 48/52 on a large sample, stop. The experiment is invalid until you explain the split.
  20. Novelty and interferenceDay-one spikes fade, and my treatment can change your outcome. Short tests and independent-user math both lie here.
  21. CUPEDSubtract the predictable part of a metric using pre-experiment data. You get the same MDE with less traffic — if the experiment was valid.

Growth and monetization

  1. Aha momentsActivation is the early action that predicts retention — found with cohort analysis, not a brainstorm.
  2. Activation and onboardingAha moments, time-to-value and onboarding flows that turn signups into active users.
  3. Time to valueHow long a new user waits to feel the outcome they came for — and the patterns that shrink that wait.
  4. Onboarding patternsChecklists, empty states, tours and walkthroughs — which surfaces actually move people toward value.
  5. Activation funnelsTreat activation as a queryable funnel, find the biggest leak, and experiment there with retention as the guardrail.
  6. The leaky bucketAcquisition without retention only scales the leak. Improving churn compounds; buying more traffic does not change the hole in the bucket.
  7. Retention curvesThree shapes matter — decay to zero, flatten above zero, smile. Flattening is the product-market-fit signal. Compare cohorts, not blended actives.
  8. Retention and engagementRetention curves, cohort analysis, DAU/MAU and why retention is the foundation of growth. Start here for the map.
  9. Engagement metricsRetention asks whether they come back. Engagement asks how deeply they use it. DAU/MAU, L7 and power-user curves only work if you match the product's natural frequency.
  10. Retention leversHabit loops, careful notifications, resurrection, critical-path quality and — for paid products — splitting voluntary vs involuntary churn. Diagnose the curve before you add an engagement feature.
  11. Funnels vs loopsFunnels diagnose steps. Loops explain where the next user comes from — the output of one cycle becomes the input of the next.
  12. Growth loops and acquisitionFunnels vs loops, virality math, content and paid loops, and referral programs.
  13. K-factorK = invites per user × conversion. Below 1 still cuts CAC. Cycle time sets how fast generations stack.
  14. Types of growth loopsContent, paid, sales and collaboration loops fit different products. Network effects amplify a loop; they are not themselves a loop.
  15. Referral programsA designed viral loop. Ask activated users, reward in-product currency, measure retention — and budget for fraud.
  16. Value metricsCharge for a unit that grows with customer value and with your cost to serve.
  17. Good-better-best packagingThree tiers so buyers self-segment. Gate admin and scale, not the aha.
  18. Pricing and packagingValue metrics, tiers, freemium vs trials, usage-based pricing and how to test prices.
  19. Freemium vs trialsFree forever when free users create distribution. Time-boxed trials when each free account is expensive to serve.
  20. Usage-based pricingBills that scale with consumption, plus the metering UX that stops a surprise invoice from destroying trust.
  21. Willingness to paySurvey ranges, interviews, and careful tests — then grandfather existing customers when you raise prices.
  22. Product-led growthThe product acquires, converts and expands. Sales does not create demand from zero for every deal.
  23. Sales-led growthHumans drive the path to revenue when ACV and buying complexity can pay for their time.
  24. Product-led vs sales-led growthWhen self-serve wins, when you need a sales team, and how hybrids work.
  25. Product-qualified leadsUsage predicts buy intent. Sales arrives with context, instead of restarting from a cold form fill.
  26. GTM and the roadmapPLG prioritizes activation and billing. Sales-led prioritizes SSO, audit logs and SE tooling. Hybrids need both.
  27. GTM failure modesTwo funnels that do not talk, sales habits inside a PLG product, and comp plans that fight the motion.

Shipping and specialties

  1. Launch tiersMatch launch ceremony to blast radius, not to how hard the code was.
  2. Alpha and betaA bounded group of users who find cliffs before you spend the marketing budget.
  3. Launching products and featuresLaunch tiers, betas, checklists and measuring whether a launch worked.
  4. Soft launch and percentage rolloutMake the feature available, learn from real load, and spend the marketing budget only after the path works.
  5. Launch checklists and runbooksA go/no-go gate across engineering, docs, support and GTM, plus a one-page launch-day clock.
  6. Measuring launchesWrite success metrics and guardrails before launch day, then double down, iterate or sunset.
  7. Feedback channelsEvery channel lies in a different way. Combine support, in-app, sales and behavior — never let volume from one source set the roadmap.
  8. Iteration and feedback loopsTurning feedback, support tickets and data into continuous improvement and kill decisions.
  9. Quantifying feedbackA lightweight taxonomy plus counts and a linked metric turns anecdotes into decisions — and stops you staffing the wrong team.
  10. Closing the feedback loopTell people what you did with their input — shipped, planned, or no — or they stop talking and you lose the channel.
  11. Iterate or killDouble down, iterate, or sunset. Killing is underused. Endless polish on a bad bet is overused.
  12. Developer experienceDevelopers adopt a path from curiosity to production, measured in time-to-first-call, not endpoint count.
  13. API design as UXResource names, errors, versioning and webhooks are the interface. Prefer additive compatibility.
  14. Developer and API productsTreating APIs, SDKs and docs as products: developer experience and adoption.
  15. SDKs, quickstarts and docsRaw HTTP is the contract. The kit around it — snippets, guides, samples, CLI — is how most people integrate.
  16. Developer funnelsA north star that reflects production value, and the cliffs between signup, first call and live traffic.
  17. API pricing and rate limitsMeter names people understand, a free tier that reaches aha, and 429s that come with docs and a dashboard.
  18. When AI fitsPick jobs with high value, tolerable errors and a verifier. Skip demos that cannot be checked.
  19. Copilot vs agent vs automationPositioning is a product choice. Assist, act with tools, or run in the background — each implies different UX and liability.
  20. AI product strategyWhat to build with AI, how to defend it, how to price it, and how evals become product requirements.
  21. Evals as product specsIf you cannot define good, you cannot ship. An eval set is how quality becomes a requirement instead of a vibe.
  22. AI product economicsInference COGS can invert a seat price. Meter recognizable work, not tokens, and cap the power users.
  23. AI product moatsFoundation models commoditize. Defend with workflow, data, integrations and distribution — not a clever prompt.
  24. Product sense frameworkClarify, pick a north star, segment, name one pain, generate three distinct solutions, then choose an MVP with metrics. Interviewers reward decisions, not idea lists.
  25. Product sense interviewsStructured answers to product design, improvement and metrics questions. Start here for the map.
  26. Diagnosing a DAU dropVerify the dashboard, scope the cut, split acquisition vs retention, then rank hypotheses. Jumping to a home-feed redesign is how candidates fail.
  27. Product sense practiceJudgment is choosing and cutting. Use company-native strengths, timed mocks and weekly 15-minute app critiques — not cramming frameworks the night before.

Startup economics and business

How startups make money: markets, business models, unit economics, financial statements, SaaS metrics and go-to-market.

Startup fundamentals

  1. Compound growthWhy a small weekly rate beats a big absolute number, and how to convert weekly growth to monthly.
  2. What a startup isStartups are companies designed to grow fast. That one constraint explains funding, equity, and why most of them fail.
  3. Startup vs lifestyle businessVenture-scale startups optimize for outcome size. Lifestyle and small businesses optimize for profit and control. Pick the game on purpose.
  4. The startup lifecycleIdea, search for fit, trough, inflection, scale, maturity. Scaling spend before fit is how well-funded teams die.
  5. Startup risk and equityMost startups return nothing to common shareholders. Equity is a skewed bet, not a signing bonus.
  6. Organic startup ideasGood ideas come from lived problems, not from a whiteboard labeled "Uber for X."
  7. Schlep blindnessFounders skip thankless, messy work. Markets often underprice exactly those problems.
  8. Tarpit ideasAttractive, crowded shapes — social networks, dating apps, Uber for X — that keep attracting founders and rarely succeed.
  9. Founder-market fitAn unfair reason to win this market — lived experience, distribution, or skill — often beats a fashionable idea.
  10. Startup ideas and why nowWhere good ideas come from, which shapes to avoid, and how to judge an idea before you quit your job.
  11. Why nowA real reason this was hard recently and is newly tractable — tech, regulation, behavior, cost, or infrastructure.
  12. What product-market fit meansA good market plus a product that satisfies it. One without the other is a hobby or a stall.
  13. Measuring product-market fitRetention cohorts are behavioral truth. The Sean Ellis test is a leading survey proxy. Use both.
  14. Product-market fitWhat fit feels like, how to measure it, the false positives that fool teams, and what changes after you find it.
  15. Product-market fit false positivesFounder heroics, friends, launch spikes, press, and one logo can look like fit and are not.
  16. Before and after product-market fitSearch stays small and measures learning. After fit you earn the right to hire, systematize go-to-market, and watch efficiency.

Markets and business models

  1. TAM, SAM and SOMThree nested revenue numbers. TAM is everyone who could pay. SAM is who you can serve now. SOM is who you can win soon.
  2. Market sizing: TAM, SAM and SOMTop-down vs bottom-up market sizing with a worked example, and how to present a number investors will trust.
  3. Bottom-up market sizingCount customers, multiply by the price you will actually charge, then filter to SAM and SOM.
  4. Presenting market sizeA credible market slide shows the formula, the source, the filters and a SOM that matches the plan.
  5. Revenue qualityWhy a recurring, expandable, high-margin dollar is worth more than a one-time or thin-margin dollar.
  6. Business modelsThe main ways software companies make money, and the economics each implies for margin, sales motion and venture scale.
  7. Customer segment economicsConsumer, SMB, mid-market and enterprise buyers change CAC, churn, sales cycle and org design inside the same model.
  8. Five ways to $100M ARRChristoph Janz's animals: elephants, deer, rabbits, mice and flies — customer count times ACV.
  9. Hybrid business modelsAlmost every large company mixes revenue lines by year ten. Start with one primary model, then add a second that reinforces it.
  10. Choosing a business modelFour questions that pick a model, what each model forces you to build, and how pricing is just packaging.
  11. Subscription contractsMonthly vs annual, ACV vs TCV, and why a signed year of cash is not the same as recognized revenue.
  12. Land and expandStart with one team, then grow seats, tiers, and modules. That is how net retention climbs above 100%.
  13. SaaS economicsSpend to acquire a customer, deliver at high margin, earn them back over many months, and expand the account.
  14. SaaS cost structureCOGS, S&M, R&D, and G&A as percentages of revenue, and why early S&M above 100% can still be rational.
  15. The SaaS cash troughYou pay to acquire a customer now and collect over many months. Growth deepens the hole until payback.
  16. GMV and take rateGMV is dollars transacted. Net revenue is GMV times the take. Never compare a GMV number to SaaS ARR.
  17. Marketplace liquidityLiquidity is the chance of a good match, fast. Buyers will not come without sellers. Density is local.
  18. Marketplace economicsTake rates, liquidity, chicken-and-egg dynamics and why some marketplaces become giants.
  19. Rake and disintermediationA take rate that exceeds the value of convenience, trust and demand invites bypass. Optimal rake funds liquidity without training people to leave.
  20. Managed vs pure marketplacesA pure platform matches and steps aside. A managed marketplace inserts ops. You trade software margins for quality and a faster cold start.
  21. Consumer unit metricsARPU and DAU/MAU — the two numbers that tell you whether a consumer product can ever pay for itself.
  22. Ad-supported economicsRevenue ≈ DAU × impressions × CPM / 1000. Ads only work at huge scale, and only if users stay.
  23. Consumer business economicsAds, subscriptions and virality: the economics of consumer apps.
  24. Consumer subscriptionsTiny ACV, low conversion, store fees and emotional churn. Retention and organic growth do the work paid ads cannot.
  25. Consumer viralityPaid acquisition is an auction against every other app. Organic loops and retention are usually the only affordable growth at consumer ARPU.
  26. Consumer retentionAcquisition can be bought. Retention usually cannot. Cohort charts beat aggregate DAU, and payer retention drives LTV.
  27. Hits and platform riskConsumer is hit-driven and platform-dependent. Plan for algorithm, fee and ranking shocks — and own the user relationship.
  28. Usage-based pricingCustomers pay for what they consume. Spend tracks value — and your costs often track usage too.
  29. Infrastructure COGSWhen you sell infrastructure you often resell someone else's, plus software on top. Margin comes from the software, not the cheapest pass-through.
  30. API, infrastructure and usage-based economicsUsage-based pricing, COGS-heavy businesses and the economics of AI API products.
  31. Developer land, enterprise expandSelf-serve API key first, then usage growth, then procurement, commits and expansion. The product must serve both ends.
  32. AI wrapper economicsAn app that only resells tokens is a reseller. Durable value sits above the model, and unlimited plans light money on fire.
  33. Commits, credits and usage packagingAnnual commits, prepaid credits, platform fees and tiered prices turn volatile usage into something a CFO can budget.
  34. Usage metering and NRRWhat you count is the business. Bad meters leak revenue or enrage customers, and usage NRR hides dollar churn inside retained logos.

Unit economics and finance

  1. Customer acquisition costFully loaded sales and marketing spend divided by new customers. Blended and paid CAC tell different stories.
  2. Contribution marginRevenue minus the variable cost of serving it. That leftover is what pays back CAC.
  3. Unit economics: CAC, LTV and paybackWhether one more customer makes money, how long until cash comes back, and why the LTV formula lies if you treat it as truth.
  4. Lifetime valueGross profit you expect from a customer over their life. The simple formula lies if churn or ARPA is not stable.
  5. CAC paybackHow many months of contribution it takes to recover CAC. Cash cares about this more than about LTV.
  6. LTV to CACThe famous 3× heuristic, why Gurley distrusted the formula, and when to optimize payback instead.
  7. COGS vs opexCOGS is the cost of delivering the product to customers. Opex is the cost of building and running the company.
  8. Gross margin and COGS, including AIWhat belongs in COGS vs opex, why classic SaaS runs ~75–80% gross margin, why AI products often sit at 50–60%, and the levers that move the number.
  9. Inference as COGSTokens paid per request are COGS. Averages hide power users who can make a flat subscription unprofitable.
  10. Gross margin leversRouting, caching, pricing, commits and product caps move AI and cloud margin. Raising price, cutting COGS and capping usage are different bets.
  11. Income statement linesRevenue minus COGS is gross profit. Minus S&M, R&D and G&A is operating income. Headcount dominates the opex lines.
  12. Revenue recognitionSaaS revenue is earned as you deliver access, not when the invoice is paid. Annual prepays become deferred revenue.
  13. The P&L for startupsReading and building an income statement — revenue recognition, COGS, S&M / R&D / G&A, EBITDA and net income — with a worked $5M ARR SaaS example.
  14. EBITDA and net incomeGross profit is after delivery. Operating income is after opex. EBITDA adds back D&A. Net income is after interest and tax.
  15. A $5M ARR P&LA growth SaaS at $5M ending ARR can show $4.2M revenue, 78% gross margin, and a multi-million operating loss — and still be on plan.
  16. Profit versus cashAccrual books record when value is earned. The bank records when money moves. Companies die from the second gap.
  17. The balance sheetA snapshot at a point in time: Assets = Liabilities + Equity. Cash and deferred revenue live here.
  18. The cash flow statementThree sections — operating, investing, financing — that explain why cash changed between two balance sheets.
  19. Cash flow and the three statementsWhy profit ≠ cash, how the balance sheet and cash flow statement work, deferred revenue from annual prepay, and how the three statements link.
  20. Deferred revenueCash collected for service not yet delivered. SaaS's cash superpower, and a trap if you spend it like profit.
  21. Working capital and receivablesAR, AP and payment terms can fund growth or starve it. Days sales outstanding is a cash number, not a bookkeeping footnote.
  22. ARR, billings and cashFour related top-lines — ARR, billings, collections and recognized revenue — that diverge when customers prepay.
  23. Operating vs financing cashThe bank can go up because you raised a round while the business still drains cash. Separate the two.
  24. Net burn and runwayGross burn is cash out. Net burn is cash out minus cash in. Runway is cash divided by net burn.
  25. Default aliveOn your current trajectory — without assuming a future raise — do you reach profitability before cash runs out?
  26. Burn, runway and default aliveCalculating burn and runway, the default-alive test, and how to extend runway.
  27. Extending runwayHiring pace, pricing, collections and cost cuts — the levers that lengthen the fuse, and a worked cut plan.
  28. Ramen profitabilityThe company covers the founders' modest living expenses. Not venture-scale profit — a powerful option.
  29. The fatal pinchRaising with short runway destroys leverage. Plan 18–24 months after a round, and start the process before you are desperate.
  30. Driver-based modelsA model is a machine that turns a few operational inputs into P&L, cash and runway. Drivers can be wrong. Plugs cannot be debugged.
  31. Revenue build and cohortsNew ARR is funnel × ARPA. Ending ARR adds expansion and subtracts churn. Capacity constraints keep the funnel honest.
  32. Building a startup financial modelA driver-based model: revenue drivers, cohorts, hiring plan and scenarios.
  33. Hiring plan in the modelPeople are most of burn. Model start months, fully loaded cost and ramp — then see what the hire does to runway.
  34. Scenarios and sensitivityOne structure, three cases. Then shock one driver at a time and see which fight is worth having.

Metrics and go-to-market

  1. ARR and MRRThe stock of recurring subscription dollars. What belongs in the number, and what must stay out.
  2. The ARR bridgeStarting ARR plus new and expansion, minus contraction and churn, equals ending ARR.
  3. Bookings vs billings vs revenueSigned, invoiced, and recognized are three clocks. Celebrating the wrong one is a sugar high.
  4. Logo churn vs revenue churnCounting lost customers and counting lost ARR diverge when account sizes differ. Always name the period.
  5. SaaS metrics: ARR, churn and net retentionARR and the bridge, bookings vs revenue, logo vs revenue churn, GRR, NRR, and the quick ratio.
  6. GRR and NRRGross retention ignores expansion and is capped at 100%. Net retention includes expansion and can exceed 100%.
  7. SaaS quick ratioHow many dollars of new and expansion ARR you add for each dollar you lose to contraction and churn.
  8. Burn multipleNet burn divided by net new ARR. How many dollars of cash did we spend to add one dollar of recurring revenue?
  9. Efficiency metrics: burn multiple, magic number and rule of 40How boards and investors judge growth efficiency, with formulas and benchmarks.
  10. SaaS magic numberAnnualized quarter-over-quarter ARR growth divided by prior-quarter S&M. Should we add sales capacity?
  11. Company-level CAC paybackHow many months until gross profit from newly acquired ARR pays back S&M. Gross margin is in the denominator.
  12. Rule of 40Revenue growth percent plus profit margin percent. Declare which profit metric you use. Necessary, not sufficient.
  13. T2D3 and efficiency tradeoffsTriple-triple-double-double-double is a growth-shape mnemonic. Capital markets decide how strictly efficiency is graded.
  14. ICP and personasThe ICP is the firm that buys, renews and expands with least friction. Personas are the people inside that firm who use, champion, buy or block.
  15. GTM motionsFour families — PLG, inside sales, field sales and partners — cover most software companies. Mix them by segment only if you split the metrics.
  16. Go-to-market strategyICP, positioning, motions and channels: how a product reaches the people who will pay.
  17. Matching motion to ACVACV is the gravity well of GTM. A $3k product cannot pay for field sales. A six-month integration cannot be self-serve.
  18. Beachhead strategyWin a narrow segment until you are the obvious default, then knock into adjacent segments that reuse the same proof.
  19. GTM funnel metricsWork backwards from the customer goal to the emails you must send. Pipeline coverage, conversion, cycle length and win rate debug the motion.
  20. Channel-market fitA channel is a repeatable path from awareness to a customer. Most startups get zero that work. One is enough.
  21. Distribution and acquisition channelsSEO, content, paid, partnerships, community and outbound: finding channels that scale.
  22. Acquisition channelsA catalogue of how demand shows up — time-to-learn, cost shape and when each channel fits. Use it as a brainstorm list, not a to-do list.
  23. The Bullseye frameworkBrainstorm many channels, test a few cheaply, then focus on the one that works. Give every test a kill criterion.
  24. Channel saturationEvery channel has a ceiling. When marginal CAC rises above payback, improve conversion, raise LTV, or run Bullseye again.
  25. Owned audiencesEmail, docs, community and open source compound. Rented channels can be turned off overnight. Engineering-as-marketing is a channel too.
  26. Outbound prospectingBuild a list of ICP accounts, earn a meeting with a short specific message, and treat volume × conversion as the plan.
  27. Sales discoveryA discovery call is a structured interview, not a demo. BANT and MEDDIC tell you whether a real deal can exist.
  28. Founder-led salesSelling before you have a sales team: discovery calls, pipeline, closing and the first sales hires.
  29. Pilots and closingDemo their workflow. Bound the pilot with metrics and a decision date. Ask for the order. Trade term for discount, not naked price cuts.
  30. First sales hiresHire sales when you can describe the repeatable motion. An AE multiplies a playbook. A VP from big-co often imports the wrong one.

Building the company

  1. What a moat isA structural advantage that protects returns. Features are not moats — they can be copied.
  2. Competition and moats7 Powers, network effects, switching costs and aggregation theory, and what defends an AI company.
  3. 7 PowersHamilton Helmer's taxonomy of durable advantage — benefit plus barrier — and the order startups usually unlock them.
  4. Network effectsNot every network is the same: direct, indirect, data, local and marketplace liquidity. Diagnose before you claim one.
  5. Aggregation theoryAggregators own demand and commoditize supply. Suppliers to someone else's aggregator negotiate from weakness.
  6. Moats for AI companiesAssume the model commoditizes. Durable advantage sits in data flywheels, workflow, distribution, evals and cost structure.
  7. First hiresHire generalists for the bottleneck. Specialists amplify a machine that already works. The test is which scarce skill unlocks the next learning loop.
  8. Founder recruitingAt seed, candidates bet on founders. Source outbound, move fast, sell the real job, and close yourself.
  9. Hiring and early teamsFirst hires, compensation bands, equity grants by role, and culture.
  10. Early compensation and equityEarly startups trade cash for upside. Use bands, standard vesting and honest dilution — not ad-hoc negotiation.
  11. Firing and onboardingThe hire is not done when they sign. Clear 90-day ownership converts equity into velocity. Wrong hires need fast feedback and a short exit.
  12. How to read SEC filingsS-1, 10-K, 10-Q, 8-K, and the proxy. A reading order that saves hours, starting from EDGAR.
  13. GAAP vs non-GAAPAudited statements versus management's adjusted picture. Stock-based compensation is the usual gap.
  14. SaaS metrics in filingsARR, RPO, NRR, large-customer counts, and margin — plus the definition box that makes or breaks them.
  15. Reading company financialsHow to read a 10-K or S-1 so you can judge an employer, a customer, or a competitor — not so you can become an analyst.
  16. Reading an S-1Snowflake's IPO prospectus as a worked example of product revenue, RPO, NRR, and large-customer counts.
  17. Premature scalingGrowing spend faster than you have a repeatable way to create value. The signature failure of well-funded teams.
  18. Co-founder conflictBreakups kill early companies. Explicit roles, vesting, and hard conversations prevent most of them.
  19. Why startups failPremature scaling, no market need, co-founder conflict, and running out of money: the patterns under the official cause of death.
  20. When to pivotA structured change of customer, job, or model that keeps what you learned. Not a rebrand every quarter.
  21. Shutting down a startupStop with eyes open. Employees, customers, and investors first. Silent abandonment burns the reputation you will need later.

Venture funding and company building

How venture capital works and how to raise it: SAFEs, priced rounds, term sheets, cap tables, employee equity and exits.

How venture works

  1. How venture capital worksLPs, GPs, fund structure, management fees, carry and the returns VCs need.
  2. VC fund structureLPs commit, GPs decide, the management company runs the firm, and a 10-year clock shapes every investment.
  3. 2 and 20The classic VC fee split: about 2% a year to run the firm, 20% of profits as carry if the fund works.
  4. TVPI, DPI and IRRWhat LPs need from a fund, and why paper value (TVPI) is not cash they can spend (DPI).
  5. Reserves and ownership targetsWhy funds keep powder for follow-ons, aim for 10–20% at entry, and under-reserve at their peril.
  6. Power-law returnsMost venture deals return little; a tiny tail returns so much it pays for the rest. The shape is the point.
  7. Fund-returner mathAt 10% ownership, one company often needs a $1B exit to return a $100M fund by itself. A $50M sale is rounding error.
  8. The power lawWhy a few investments return the whole fund, and how that shapes what VCs fund.
  9. Black swan farmingInstitutional venture optimizes for the right to win huge, not for a high hit rate of pretty-good companies.
  10. Founder and VC misalignmentA $50M exit can change a founder's life and be a shrug for the fund. Know the game before you raise.
  11. The funding staircasePre-seed through IPO is a sequence of checks that each buy down a different risk — not one round with a louder letter.
  12. Round sizes and dilutionTypical check sizes by stage, Carta's 2025 medians, and the algebra that 20% sold is 20% of everyone's stake.
  13. Round milestonesWhat proof each stage is buying — from "can this team build" through repeatable growth to public-market hygiene.
  14. Funding stages from pre-seed to IPOWhat each round is for, typical sizes, dilution and the milestones investors expect.
  15. Founder dilution across roundsHow 100% becomes the teens — pools, 15–25% primary rounds, and why landing in single digits at IPO is normal.
  16. Angel investorsPeople writing personal checks — typical sizes, what they add, and how to pick them.
  17. Syndicates and rolling fundsHow a lead packages many small checks into one line on the cap table, and how that differs from a solo angel.
  18. AcceleratorsA fixed-term program that trades a standard check and equity for network, focus and Demo Day.
  19. Angels, accelerators and YCA map of people-checks, syndicates, accelerators and the YC deal — choose them as colleagues, not ATMs.
  20. The YC standard dealTwo SAFEs that add to $500k — a fixed 7% plus an uncapped MFN — and how the second piece moves with your next cap.

Company setup and early instruments

  1. LLC vs S-corp vs C-corpWhy venture-backed companies form a Delaware C-corp, and when an LLC is still the right call.
  2. Authorized shares and founder stockHow a charter's share count, par value and restricted stock purchases turn a new C-corp into a cap table.
  3. PIIA and IP assignmentInvestors buy a company that owns its product. A PIIA is how code, models and brand move from people into the entity.
  4. Incorporation and the Delaware C-corpWhy VC-backed startups incorporate as Delaware C-corps, how founder stock is issued, and the setup that prevents diligence disasters later.
  5. Formation tools and diligence cleanupAtlas and Clerky for a vanilla Delaware setup, when to hire a lawyer, flips for non-US founders, and the eight messes Series A counsel always finds.
  6. Splitting founder equityNear-equal splits for co-founders who start together, and the few reasons an uneven split is actually justified.
  7. Reverse vestingFounders buy stock on day one, then earn the right to keep it — four years, a one-year cliff, and company repurchase of the rest.
  8. Founder equity, vesting and agreementsHow co-founders split equity, implement four-year reverse vesting with cliffs, handle acceleration and file the 83(b) before the clock runs out.
  9. Founder accelerationSingle-trigger vs double-trigger vesting — who gets unvested shares when the company is sold or a founder is fired.
  10. Co-founder agreementsRoles, IP, time commitment, departure and deadlock — the written rules that keep a 50/50 company from freezing.
  11. SAFEsThe post-money SAFE as a right to future equity — valuation caps, discounts, MFN, pro-rata side letters and the ownership math founders must run before signing.
  12. Post-money SAFEsOwnership at the cap is investment divided by cap, and later SAFEs dilute founders rather than earlier SAFE holders.
  13. SAFE caps and discountsThe valuation cap, the discount, "better of," uncapped SAFEs, and when each one actually binds.
  14. MFN and SAFE side lettersMost-favored-nation clauses and pro-rata side letters — the extras that sit beside the standard SAFE form.
  15. Choosing a SAFE capA cap is an ownership sale. Size the dollars against the cap, stay well under it, and stop when you hit your band.
  16. SAFE conversion mathStack post-money SAFEs, convert them in a priced round with an option pool, and work share counts and ownership before and after Series A.
  17. Stacking post-money SAFEsLater post-money SAFEs do not dilute earlier SAFE percentages; they dilute common, and the share math follows from that.
  18. Cap vs discount at conversionEach SAFE converts at the more favorable of its cap price and the discounted round price — lower price, more shares.
  19. SAFEs into Series ATwo stacked SAFEs, a $5M Series A, and a 10% post-money pool — founders 56%, SAFEs 14%, new money 20%.
  20. Convertible note termsPrincipal, interest, maturity, discount, cap and the qualified-financing trigger that turns a loan into stock.
  21. Convertible notesDebt that converts — when notes still beat SAFEs, and the map of terms, math and maturity.
  22. Note conversion mathBetter of discount or cap, plus accrued interest, walked through a $500k note into a $2.00 Series A.
  23. Note maturityThe due date a SAFE does not have, and the four conversations you get if no priced round arrives.

Priced rounds and term sheets

  1. Preferred vs common stockVCs buy preferred with seniority, anti-dilution and control. Founders and employees hold common — the residual.
  2. Pre-money and post-moneyPost-money = pre-money + new cash. Investor ownership = investment / post-money. Price per share falls out of that.
  3. Priced roundsPreferred stock at a negotiated price — pre- and post-money, share math, the pool shuffle and the closing binder.
  4. Walking a priced Series A$8M at $32M pre with a 10% post-money pool — share counts, price per share and the cap table after close.
  5. Closing a priced roundTerm sheet to charter, SPA and rights agreements — plus multiple closings, SAFE conversion and what to ask as an engineer.
  6. How startup valuation really worksEarly valuations as negotiated ownership, later ones as revenue multiples — and why the headline number can lie.
  7. Ownership-backed valuationAt seed and Series A, price falls out of cash needed and the slice the lead must own — not a DCF of the code.
  8. Revenue multiplesLater-stage posts are often ARR times a growth-and-efficiency multiple, with public comps as a noisy floor.
  9. Valuation vs termsA higher headline with participating prefs and a fat pool can leave common worse than a lower clean deal.
  10. Term sheets: an overviewThe anatomy of a VC term sheet — economic vs control terms, what is binding, and what to negotiate first.
  11. Economics vs controlAlmost every term-sheet fight is who gets the money or who decides — valuation is only one cell in the first column.
  12. Binding term sheet termsMost of a VC term sheet is an agreement to agree. No-shop, confidentiality, and sometimes expenses actually bind.
  13. Market term sheet termsA clean US seed or Series A baseline — 1× non-participating, weighted-average anti-dilution, sensible board, hiring-justified pool.
  14. Negotiating term sheetsFight valuation-plus-pool, preferences, and the board first — then run a thirty-minute checklist and a two-sheet comparison.
  15. Non-participating preferredThe market standard — 1× preferred takes the better of money back or converting to common.
  16. Participating preferredMoney back plus a share of the rest — costly for common, and why a cap is the usual compromise.
  17. Preference multiples2× and 3× preferences flatten common outcomes across a wide exit band — treat them as a distress term.
  18. Seniority and stacked waterfallsLater preferred can stand in front of earlier preferred. A $25M sale can still zero common.
  19. Liquidation preferences and waterfallsWho gets paid first in an exit — the map of 1× non-participating, participation, multiples and stacked seniority.
  20. Full ratchetThe harsh anti-dilution formula that resets the old conversion price all the way to the new cheap price, even on a tiny round.
  21. Weighted-average anti-dilutionThe market formula CP2 = CP1 × (A+B)/(A+C), plus why the definition of A (narrow vs broad) changes the pain.
  22. Anti-dilution protectionExtra conversion-price math for old preferred when a later round is cheaper — not ordinary new-money dilution.
  23. Pay-to-playFollow on in the next round or lose anti-dilution — and sometimes lose preferred status.
  24. Board seatsWho sits on the board at each stage, why independents matter, and how an investor-majority board fires a CEO.
  25. Protective provisionsThe preferred veto list — what must stay on it, and what turns the company into a hostage.
  26. Control termsBoard seats, vetoes, drag-along, pro-rata and ROFR — who can force a sale, block a raise, or fire a CEO.
  27. Drag-along and tag-alongHow a majority can force everyone to sell, and how a minority can join a sale they did not start.
  28. Pro-rata rightsThe right to buy enough of the next round to hold your percentage — and why super pro-rata is a scarce concession.
  29. ROFR and co-saleHow the company and investors can match a founder share sale, and sell alongside it.

Cap tables and equity

  1. Fully diluted ownershipAuthorized vs issued vs fully diluted — and why "I own 1%" is meaningless until you name the denominator.
  2. How dilution worksYour share count stays put; the denominator grows. Each round multiplies old ownership by (1 minus the new-money percent).
  3. Cap tables and dilutionThe ownership ledger from founding through Series B — start here for the map, then walk the numbers.
  4. Walking a cap tableNorthwind from incorporation through two stacked post-money SAFEs, with the ownership math at each step.
  5. Series A on the cap tableConvert SAFEs, refresh the pool, sell preferred, then watch Series B and pro-rata move the same numbers again.
  6. Sizing an option poolBuild a hiring plan, sum the grants, add a buffer — do not pick 15% because a term sheet said so.
  7. The option pool shuffleA post-money pool target is usually carved from the pre-money, so founders — not the new investor — pay for future hires.
  8. Option poolsReserved equity for hiring — size it from a plan, watch the pre-money shuffle, and refresh before the tank is empty.
  9. Option pool refreshesLater rounds top the pool up again. Track available, outstanding and grant burn so you are not surprised in diligence.
  10. Options vs RSUs vs restricted stockThree ways companies grant equity — a right to buy, a promise of shares at vest, or actual shares upfront.
  11. Strike price and 409AOption strikes track a 409A appraisal of common stock, which usually sits below the last preferred round price.
  12. Vesting and accelerationFour years, a one-year cliff, refresh grants, and the difference between single-trigger and double-trigger acceleration.
  13. Exercising optionsPaying strike to buy shares, early exercise plus 83(b), and the post-termination window that can force a cash decision in 90 days.
  14. Employee equityOptions vs RSUs, strike prices and 409A, vesting and exercise windows, and how to evaluate a startup offer with ownership math.
  15. Evaluating an equity offerTurn "50,000 options" into ownership %, diluted %, exercise cost and dollars after the preference waterfall.
  16. NSO taxationNon-qualified options are taxed as ordinary income on the spread at exercise, then capital gain or loss when you sell.
  17. ISOs and AMTIncentive stock options skip regular tax at exercise, but the spread is an AMT adjustment — and a qualifying sale needs two holding periods.
  18. 83(b) electionsA 30-day election to tax restricted stock at grant instead of at vest — powerful at a tiny spread, irreversible if you forfeit.
  19. QSBSSection 1202 can exclude a large fraction of federal gain on eligible C-corp stock — with different rules for shares issued before and after July 4, 2025.
  20. Equity taxes: ISOs, NSOs, 83(b), AMT and QSBSHow US tax rules hit option exercise and share sales — NSO ordinary income, ISO AMT, 83(b), QSBS after the 2025 changes.

Raising

  1. Pitch deck slidesTen to fourteen slides in a standard order — one job per slide, numbers that survive without you talking.
  2. The pitch deckA short slide narrative that sells the investment — get the meeting, frame the story, show why now and why you.
  3. Seed vs Series A decksSame skeleton, different burden of proof — insight and early evidence versus a repeatable growth machine.
  4. Deck designLegible, simple, obvious — one idea per slide, readable on a phone, numbers over adjectives.
  5. When to raiseRaise when capital buys a milestone, with 6–9 months of runway left — not when you are merely anxious.
  6. Target lists and introsRank 40–80 names before you take meetings, and make every intro warm and forwardable.
  7. Batching a fundraiseA 2–4 week sprint of first meetings, dated partner slots and fast diligence — not a year of polite coffees.
  8. Running a fundraiseTarget lists, warm intros, a tight 2–4 week process, momentum, data rooms and closing — how a raise actually gets done.
  9. Closing a roundFirst commitment, leads vs followers, reading soft nos, collecting wires, and keeping Plan B so the process is not existential.
  10. Monthly investor updatesA short, predictable email — highlights, stable metrics, honest lowlights and specific asks — is the cultural version of information rights.
  11. Running board meetingsPre-reads 48 hours ahead, labeled decisions, recorded resolutions and a closed session — not a TED talk.
  12. Managing investors and boardsMonthly updates, board meetings, hard conversations and how to use investors for hiring and intros without losing the company.
  13. Board dynamicsHealthy boards argue about strategy. Unhealthy boards rubber-stamp — or turn every meeting into a referendum on the CEO.
  14. Communicating bad newsBad news early, complete and with a plan — never as a meeting ambush the day you need a vote.
  15. Bridges and flat roundsInsider capital and same-price extensions that buy time — and when they are just delayed insolvency.
  16. Down roundsA priced round cheaper than the last one — what happens to founders, employees and anti-dilution.
  17. Down rounds, bridges and venture debtA map of flats, bridges, clean downs, recaps, debt and structured "flats" — what to do when the last price will not hold.
  18. RecapitalizationsA new financing that reshuffles rights — senior money in, old preferred cut down, and pay-to-play for those who follow.
  19. Venture debtA loan on top of equity — size, warrants, covenants, and why it is a poor rescue tool.
  20. Structured roundsFlat or up headlines that hide extra preference, warrants and guaranteed returns — often worse than a clean down round.

Exits and alternatives

  1. Why companies buy startupsTalent, product, customers or defense — the buyer's motive predicts the deal structure and who gets paid.
  2. The acquisition processInbound to LOI to diligence to close — and why you should not open talks unless you intend to sell.
  3. Cash, stock, earnouts and escrowHow the purchase price is actually paid — and why headline value is not what hits your account.
  4. The acquisition waterfallFees, debt, escrow and preferred get paid before common. A $40M headline can leave common $4.8M.
  5. Mergers and acquisitionsHow acquisitions work: motives, process, consideration, the waterfall and what happens to employee equity.
  6. Equity in an acquisitionUnvested options may accelerate, convert to buyer stock, or vanish — and how to read a deal as an employee.
  7. Secondaries and tender offersLiquidity before exit for founders and employees: secondary sales, tender offers and their rules.
  8. Founder secondariesSelling a modest slice of vested founder stock in a later priced round — liquidity without selling the company.
  9. Tender offersA company-sponsored window where eligible holders can sell a capped slice of shares at a fixed price.
  10. Transfer restrictionsROFR, company consent, securities law and lockups — why a marketplace match is the start of paperwork, not a sale.
  11. Secondary pricing and taxCommon often sells at a discount to last preferred, 409A is a separate number, and holding period decides the tax character.
  12. The traditional IPOUnderwriters, the S-1, the roadshow, the offering price and the first-day pop — how a company actually lists.
  13. IPO lockupsInsiders usually cannot sell for about 180 days. IPO day is not payday.
  14. IPOs and going publicThe map of listing — traditional IPOs, lockups, preferred conversion, direct listings and why companies stay private longer.
  15. Preferred stock at IPOA qualifying IPO usually converts preferred into common, so the preference stack disappears instead of eating a cash waterfall.
  16. Direct listings and SPACsTwo other doors to a public ticker — market-set opening prices without a classic book, and SPAC mergers with extra dilution.
  17. BootstrappingGrowing from revenue and savings instead of institutional equity, and what you keep and give up.
  18. Revenue-based financingAn advance you repay as a cut of monthly revenue until you hit a cap, with no equity sold.
  19. Grants and non-dilutive fundingPublic and research money that does not buy shares, and when the paperwork is worth it.
  20. Customer fundingPaid pilots, services wedges, rewards crowdfunding and community rounds that put cash in before a priced raise.
  21. Alternatives to venture capitalA map of bootstrap, RBF, grants, customer funding and debt — and how to pick the game you are playing.