TCP latency traps and UDP

TCP's guarantees are useful, but they have costs that show up as latency. Its ordered stream can stall unrelated data. Two optimizations can interact to add a fixed delay to every request. And some traffic is better off without guarantees at all, which is where UDP comes in.

Head-of-line blocking

TCP promises an ordered byte stream. Suppose packet 5 of 20 is lost. Packets 6 to 20 may already sit in the receiver's kernel, but the application cannot read any of them. It waits until packet 5 is retransmitted, which costs at least one round trip.

That is harmless for a single file download. It is painful when many independent streams share one connection, as in HTTP/2. One lost packet stalls every stream. That is the main reason HTTP/3 moved to QUIC over UDP (see HTTP/3 and QUIC).

The Nagle and delayed-ACK trap

Nagle's algorithm dates from 1984. It targets "tinygrams", which are tiny packets. If unacknowledged data is in flight, TCP buffers small writes until the ACK arrives or a full segment builds up. Combined with the receiver's delayed ACK, which waits up to 40 ms on Linux (200 ms on some stacks) to piggyback an ACK, it creates a classic latency bug.

Here is the sequence:

1. The client writes a request in two small pieces: headers, then body.
2. The second piece waits, because the first piece is not yet acknowledged.
3. The server delays that ACK, hoping to send data with it.
4. Every request picks up a mysterious 40 ms.

The symptom is a p50 or p99 that sits at exactly 40 ms or 200 ms. The fix is to set TCP_NODELAY on the socket, which turns Nagle off. Most modern runtimes and RPC libraries already do this. Go sets it on every TCP connection by default. If you write your own protocol on raw sockets, set it yourself, and combine small writes into one call where you can.

conn, err := net.Dial("tcp", "db.internal:5432")
if err != nil {
	return err
}
// Go already defaults to true; shown for clarity.
conn.(*net.TCPConn).SetNoDelay(true)

UDP: no guarantees, on purpose

UDP adds only ports and a checksum to IP. There is no handshake, no ordering, no retransmission and no congestion control. A datagram arrives once, or not at all. That sounds worse, but it is exactly right when TCP's guarantees cost more than they are worth:

• DNS. A query and its answer each fit in one packet. A TCP handshake would triple the cost of a lookup. If a packet is lost, the client asks again after a short timeout. DNS falls back to TCP for large responses.
• Real-time media and games. A video frame or a player-position update that arrives 300 ms late is useless. Dropping it and rendering the next one beats stalling the stream.
• QUIC. HTTP/3's transport runs on UDP so it can implement its own per-stream reliability and congestion control in user space. It avoids TCP's head-of-line blocking, and it avoids waiting years for kernels and middleboxes to accept a new transport protocol.
• Telemetry. StatsD metrics can lose a sample without harm, and the sender must never block.

The cost is that you inherit every problem TCP solved. If you need reliability over UDP, use QUIC rather than inventing your own. Some corporate networks block or throttle UDP, so anything built on it needs a TCP fallback. Browsers fall back from HTTP/3 to HTTP/2 automatically.

Common mistakes

• Leaving Nagle on for request-response traffic. Small writes then wait for ACKs that are delayed.
• Using UDP for data that must arrive. You will rebuild TCP badly. Use TCP or QUIC.
• Assuming UDP always works. Some networks drop it, so keep a fallback.

Check yourself

• Why does a request show a fixed 40 ms delay, and what turns it off? Nagle waits for an ACK that the server delays. TCP_NODELAY disables Nagle.
• Why is UDP a good fit for DNS? One query and one answer each fit in a packet, so the handshake would cost more than the lookup.

Key takeaways

• TCP's ordered stream causes head-of-line blocking. One lost packet stalls everything behind it.
• Nagle and delayed ACK together can add a fixed 40 ms (or 200 ms) to small request-response writes. TCP_NODELAY removes Nagle.
• UDP suits short exchanges and loss-tolerant data, such as DNS, live media and telemetry.
• If you need reliable delivery over UDP, use QUIC, not a home-grown protocol.

Further learning

Next, use real tools to see handshakes, retransmissions and windows on your own machine, in network tools.

For how QUIC fixes the stall described above, see HTTP/3 and QUIC.