Cloud load balancers

A load balancer stands in front of several servers and spreads incoming connections across the healthy ones. Cloud providers offer several kinds. Most of the choice comes down to one question: do you need to read the request, or only move the bytes?

The big idea

Think of a host at a restaurant. Guests are seated only at tables marked open. A table that is closed for cleaning gets no guests until it reopens. The host does not cook, and does not decide what you order.

The analogy hides one thing. A real load balancer also has to hold connections open and close them cleanly. The metaphor covers routing, not the connection handling.

The types

Target groups hold the backends. Health checks decide which targets get traffic.

On GCP, the global external HTTP(S) load balancer plays the ALB role. It uses an anycast front IP, URL maps and backends in several regions.

Walking through a health check failure

Say three API instances sit behind a load balancer, and the health check calls /healthz. That endpoint runs a database query.

1. A short database blip makes the query fail.
2. All three instances fail the health check at once.
3. The load balancer has no healthy target left, so it returns errors to every client.
4. The instances were fine. The check made them look dead.

The fix: use a shallow check for load balancer health, such as "the process is up." Put dependency checks in your monitoring, not in the traffic path.

Cross-zone balancing

Cross-zone load balancing sends traffic to targets in any availability zone. On ALB it is usually on by default, which evens out load when zones have uneven target counts. NLB's default has differed in the past, so check your setting. Cross-zone traffic can add inter-AZ data charges.

Behind the load balancer: Ingress and Gateway API

Inside a cluster, a Service of type LoadBalancer gets its own cloud load balancer. That is simple for a few entry points and expensive at scale. An Ingress controller instead shares one or a few load balancers across many hosts and paths. Gateway API is the newer model. It splits roles: infrastructure owns the Gateway, and app teams own the routes.

Wrong target settings break things quietly. A target type that points at nodes instead of pods can lose the client IP, or fail health checks.

Common mistakes

• Health checks that run a dependency query. They can remove the whole fleet.
• Using NLB for every API. You lose L7 routing for no reason.
• Forgetting cross-zone charges. Balancing evenly can cost more.
• Copying controller annotations from old answers. Read the docs for your controller first.

Check yourself

1. You need a static IP for a PrivateLink entry point. Which type fits? An NLB. It offers static IPs and PrivateLink, which the ALB does not.
2. A health check runs a database query, and the database blips briefly. What happens? Every target can fail at once, leaving no healthy backend, so clients get errors.

Key takeaways

• L7 balancers read HTTP. L4 balancers move connections.
• Health checks should be shallow, so a dependency blip does not remove the fleet.
• Ingress and Gateway API share one entry point across many services.

Further learning

Next, see how DNS decides which load balancer a name points at in DNS routing and TTL.