Service communication: synchronous vs asynchronous

Services talk in two basic styles. In synchronous request and response (REST or gRPC, see API design), the caller waits for an answer. In asynchronous messaging, the caller publishes an event through a broker and moves on. Sync is simple and gives an immediate answer, but it couples availability. If Orders calls Payments and waits, Orders is down whenever Payments is down. Async decouples timing and availability. The cost is eventual consistency and harder debugging.

The big idea

Sync calls compound. Each call in a chain is another thing that can fail. An async event breaks the chain: the producer's job is done once the event is accepted. Use async when the caller doesn't need the answer to finish its own work. "Order placed, tell billing and shipping later" is a good example.

Walking through it

Five services in sequence, each 99.9% available:

availability = 0.999^5 = 0.995 -> 99.5%
downtime     = 0.5% x 43,200 minutes per month = about 216 minutes
compare: one 99.9% service, about 43 minutes per month

Latency adds up the same way. A request that fans out to many services is only as fast as its slowest call, and the tail latency of each call compounds.

Async keeps the producer fast and independent:

Checkout:  save order 881 (own database)   publish OrderPlaced(881)   return
Billing:   consume OrderPlaced(881)        charge the card             (later)
Shipping:  consume OrderPlaced(881)        create the label            (later)

Checkout returns even if Billing is down. The message waits in the queue until Billing is back.

The distributed monolith

The most common failure is the distributed monolith: services that are deployed separately but can't be changed separately. Its smells:

• Shipping one feature needs coordinated releases of several services, in a fixed order.
• Services call each other many times per request, often in long synchronous chains.
• Services share a database, or a library of domain models that must be upgraded in lockstep.
• One service's outage takes down the whole product.

A distributed monolith has the costs of microservices, with network failures and operations work, and the coupling of a monolith. If you see these smells, merge the services or redraw the boundaries.

Boundaries have a price

A real case shows what network boundaries cost. In 2023, an Amazon Prime Video team described a tool that monitors video stream quality. The first design used AWS Step Functions to orchestrate Lambda functions, with video frames passed between components through S3. At scale, the state transitions and S3 transfers became expensive and hit scaling limits. Moving the components into one process on ECS, with frames passed in memory, cut that workload's infrastructure cost by over 90%. The lesson is specific. When components exchange large volumes of data at high frequency, network boundaries are expensive. It was one workload's design, not a retreat from services.

Common mistakes

• Long synchronous chains. Each hop multiplies the failure odds. Keep chains short, and use events for steps the caller doesn't need to wait for.
• Chatty services. Many calls per request usually mean the boundary is in the wrong place.
• Assuming async means no coupling. Consumers depend on the shape of events, so version them like APIs.
• Moving bulk data across a network at high frequency. Keep it in one process when you can.

Check yourself

1. Five services in a chain, each 99.9% available. What is the availability?

   About 99.5%, which allows roughly 216 minutes of downtime a month.

2. When is async the better choice?

   When the caller doesn't need the result to finish its own work, like telling billing about an order that is already saved.

Key takeaways

• Sync is simple, but it couples availability across a whole call chain.
• Async decouples timing, and the caller can finish its work without the consumer.
• A chain of synchronous calls multiplies unavailability and latency.
• A distributed monolith has the costs of both approaches, so redraw its boundaries.

Further learning

Next: database per service, which covers who owns the data.