Connection reuse

Run the same request again a few seconds later, over the same connection. It is much faster. Nothing about the server changed. The DNS lookup, the TCP handshake, the TLS handshake and the slow start were all paid for the first time, and the second request skips them. Reusing a connection is one of the highest-leverage defaults in backend code.

Walking through it: cold vs warm

Cold request:   DNS + TCP + TLS + request/response + slow start   ~890 ms
Warm request:   request → first byte (1 RTT + 60 ms)              260 ms
                body fits in one flight (window already grown)      0 extra

No DNS lookup, since it is cached. No TCP handshake. No TLS handshake. No slow start, because the congestion window has already grown. The 260 ms that remains is the round trip to Virginia plus the server's work.

Where reuse appears

• Browsers keep connections alive and reuse one HTTP/2 or HTTP/3 connection for every resource on an origin. They also share connections across hostnames covered by the same certificate.
• CDNs keep warm pools to your origin, as described in latency budget and round trips.
• Your services should share one long-lived HTTP client with a connection pool. Creating a client per request pays the handshake every time. In Node, use a keep-alive Agent. In Python, use a requests.Session or a shared httpx.Client.
• Database pools such as PgBouncer and HikariCP exist because a new Postgres connection costs a TCP handshake, a TLS handshake, authentication and a forked backend process. That is several milliseconds even inside one region.

Walking through it: a Go client that pools

Here is a sketch of a shared client in Go. It needs the context, io, net/http and time packages:

// One client for the whole process. Its Transport pools connections per host.
var client = &http.Client{
	Timeout: 2 * time.Second,
	Transport: &http.Transport{
		MaxIdleConnsPerHost: 100,      // default is 2: far too low for busy services
		IdleConnTimeout:     90 * time.Second,
	},
}

func fetchRecs(ctx context.Context, id string) ([]byte, error) {
	req, _ := http.NewRequestWithContext(ctx, "GET", "http://recs.internal/v1/recs/"+id, nil)
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	return io.ReadAll(resp.Body) // read fully so the connection returns to the pool
}

Two details matter. Reading the body to the end lets the connection go back into the pool. Setting MaxIdleConnsPerHost above the default of 2 keeps busy services from opening new connections all the time.

The trap: idle timeouts

If the server closes idle connections sooner than the client expects, the client tries to reuse a dead connection and gets a reset or a 502. The fix is to keep the client's idle timeout shorter than the server's and the load balancer's idle timeouts.

Common mistakes

• One client per request. Every call pays the handshake again.
• Leaving the default pool size. Busy services hit the limit and open new connections under load.
• Not reading response bodies. The connection cannot return to the pool until the body is read to the end.

Check yourself

• Why is the warm request roughly 260 ms instead of 890 ms? DNS, TCP, TLS and slow start were already paid, so only one round trip plus server time is left.
• Why do sporadic 502s appear after you add a connection pool? The client reuses a connection the server closed as idle, so its idle timeout is longer than the server's.

Key takeaways

• A reused connection skips DNS, TCP, TLS and slow start. The gain is largest when the first request is far away.
• Share one HTTP client and one database pool per process.
• Set pool sizes above the defaults for busy services, and keep client idle timeouts shorter than the server's.

Further learning

Next, see how to measure where a request's time goes once connections are in order, in measuring a request.