Speed of light and round trips

Light in optical fiber travels at roughly 200,000 km per second, about two thirds of its speed in a vacuum. That sounds fast until you put a server in Virginia and a user in Sydney, about 16,000 km apart. The one-way trip takes at least 80 ms. A round trip takes at least 160 ms, and in practice closer to 200 ms, because cables don't follow the shortest path.

The big idea

Distance sets a floor on latency. No amount of server tuning removes it. The only fix is to move the content closer to the user.

Think of ordering a pizza. A shop across the country takes a long time, however fast its ovens. A shop next door is quick, even if its ovens are slow. A CDN is the shop next door, but for your bytes and your connections.

The analogy breaks in one place. A shop can't deliver a pizza it doesn't make. A CDN can also help with requests it can't cache, by shortening the handshakes and reusing connections to your origin.

Walking through it

Here is a fresh request from Sydney to a Virginia origin, for content that can't be cached.

Direct to origin (RTT 200 ms):
  TCP handshake            1 RTT  = 200 ms
  TLS 1.3 handshake        1 RTT  = 200 ms
  Request and response     1 RTT  = 200 ms  (+ server time)
  Total                           ≈ 600 ms

Now route the same request through a Sydney point of presence (PoP), with a 10 ms user-to-PoP round trip and a warm connection from the PoP to the origin:

Through a Sydney PoP:
  TCP + TLS to PoP          2 RTT  =  20 ms
  Request to PoP                   =  10 ms
  PoP to origin, reused conn 1 RTT = 200 ms
  Total                           ≈ 230 ms

That is about 2.6 times faster for content the CDN can't cache. The PoP handles the costly handshakes over a short hop, and it reuses connections to the origin. For a cached hit, the total falls to about 30 ms.

Common mistakes

• Expecting server tuning to fix distance. A faster origin can't beat the speed of light. Move the content.
• Counting only the request. A new HTTPS connection pays for its handshakes first. Three round trips is common.
• Skipping the edge for uncacheable traffic. Even dynamic responses gain from edge TLS termination and reused connections.

Check yourself

1. Why does a fresh HTTPS request from Sydney to Virginia take about 600 ms? It needs three round trips of about 200 ms each: TCP, TLS and the request itself.
2. What does a nearby PoP save on an uncacheable request? The handshakes run over the short user-to-PoP hop, and the PoP reuses warm connections to the origin.

Key takeaways

• Light speed sets a latency floor. Only moving content closer lowers it.
• A new HTTPS connection costs several round trips before the first byte.
• Even uncacheable requests get faster through a nearby PoP.

Further learning

High Performance Browser Networking (Ilya Grigorik) explains round trips, TLS and connection reuse in detail.

Next, read about CDN routing and cache hierarchy, which shows how a CDN decides which PoP serves a user.