Autoscaling pods and nodes

Autoscaling has three layers, and each one answers a different question. The Horizontal Pod Autoscaler (HPA) changes the number of replicas. The cluster autoscaler, or Karpenter, adds nodes when pods cannot be scheduled. KEDA scales workers on event sources such as queue depth. Autoscaling without capacity planning only moves the outage somewhere else.

The big idea

Think of a restaurant on a busy night. The HPA adds waiters when the tables are full. The cluster autoscaler adds tables when there is no room left to put a waiter. KEDA adds cooks when the order queue grows. Each one needs the layer below it to have room.

Walking through it

Scaling flows from metrics to pods to nodes:

The HPA reads a metric and changes replicas to hit a target:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Three things must be true for this to work:

• The HPA computes utilization as usage divided by the request, so requests must be set.
• The Deployment should not set spec.replicas. Otherwise every apply resets the count and fights the autoscaler.
• The cluster needs room. If the HPA wants 40 pods at 1 CPU each, you need about 40 schedulable cores, plus headroom for system pods. Watch for Pending pods with Insufficient cpu during load tests. That is the signal to raise the node pool's maximum.

For nodes, the cluster autoscaler or Karpenter adds capacity when pods are unschedulable, and removes empty nodes. Karpenter provisions right-sized instances faster, and it is the common choice for new EKS fleets as of 2025 to 2026.

Vertical Pod Autoscaler adjusts requests and limits. Run it in recommendation mode first. Do not run it in automatic mode on the same resource the HPA scales on, because the two fight each other.

KEDA scales on external event sources, such as SQS depth, Kafka lag or a Prometheus query, and can scale to zero. It suits queue workers. For latency-sensitive HTTP services, keep at least one replica running, because a cold start adds delay.

Scaling flaps when CPU oscillates around the target. Set behavior.scaleDown.stabilizationWindowSeconds, often 300, so brief lulls do not cause a scale-down. For user-facing APIs, keep scale-up quicker than scale-down.

Common mistakes

• Autoscaling without requests. CPU-based HPA cannot compute utilization.
• Forgetting the node layer. Pods stay Pending, and the HPA looks broken when the cluster is the limit.
• Using CPU for an I/O-bound API. The metric stays low while requests queue.

Check yourself

1. The HPA wants more pods, but they stay Pending with Insufficient cpu. Which layer is short? The nodes. Raise the node pool maximum, or let Karpenter provision more capacity.
2. Why is CPU a poor HPA metric for an API that waits on a database? Its pods sit idle while requests queue, so CPU stays low even when users are waiting.

Key takeaways

• The HPA scales pods, the cluster autoscaler or Karpenter scales nodes, and KEDA scales on events.
• Requests must be set, and the Deployment should leave replicas to the HPA.
• Pick the metric that matches the wait: RPS or concurrency for APIs, queue depth for workers.

Further learning

Load-testing the autoscaling path is part of capacity planning. The official HPA documentation covers the full behavior options.

Horizontal Pod Autoscaling (official docs)In Kubernetes, a HorizontalPodAutoscaler automatically updates a workload resource (such as a Deployment or StatefulSet), with the aim of automatically scaling capacity to match demand. Horizontal scaling means that t…Kubernetes

Karpenter documentationJust-in-time Nodes for Any Kubernetes Clusterkarpenter.sh

KEDA documentationHow you can start with KEDA as new userKEDA