VMs, spot and autoscaling

A virtual machine (an EC2 instance or a Compute Engine VM) is the base layer for almost everything else, including Kubernetes nodes and many managed databases. Providers group VM types into families by how much memory each vCPU gets. Choosing the family and the size is mostly a question of which resource runs out first.

VM families and sizing

AWS names decode mechanically. In m7g.2xlarge, m is the family, 7 is the generation, g means Graviton (ARM), and 2xlarge is the size. Each size step doubles resources. On an m, 2xlarge is 8 vCPUs with 32 GiB.

Three rules help with sizing.

1. Start from the bottleneck. If CPU hits 80% while memory sits at 20%, move to a compute family.
2. Prefer the newest generation. Each one is usually 10 to 25% better in price-performance.
3. Try ARM. AWS claims Graviton gives up to about 40% better price-performance. Node, Python, Java and Go usually run on it unchanged.

Spot capacity

Spot (AWS) and Spot VMs (GCP) sell spare capacity at a steep discount, typically 60 to 90% off. The provider can take it back. AWS gives a two-minute warning and GCP about 30 seconds.

Spot suits stateless, interruptible work: CI runners, batch jobs, and web tiers behind a load balancer with a few on-demand instances as a floor. It is a poor fit for a single database primary.

Batch fleet: 40 x m7i.xlarge, 24/7
On-demand at ~$0.20/hour:  40 x 0.20 x 730 h = $5,840/month
Spot at ~70% off:          40 x 0.06 x 730 h = $1,752/month
Savings: about $4,090/month, if the jobs checkpoint and retry

Spread the fleet across several instance types and zones, so a shortage of one type does not drain it.

Autoscaling groups

You rarely launch VMs one by one. An Auto Scaling group (AWS) or managed instance group (GCP) keeps identical VMs running from a template. It replaces unhealthy instances and adds or removes them to hold a metric near a target.

This AWS command keeps average CPU near 50 percent.

aws autoscaling put-scaling-policy \
  --auto-scaling-group-name api-asg \
  --policy-name cpu50 \
  --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{
    "PredefinedMetricSpecification": {"PredefinedMetricType": "ASGAverageCPUUtilization"},
    "TargetValue": 50.0
  }'

The target is 50%, not 80%, because a new instance takes minutes to boot and warm up. The headroom absorbs the spike while that happens.

Common mistakes

• Running a single database primary on spot. An interruption takes the database with it.
• Buying one instance type for spot. A shortage of that type then empties the fleet.
• Targeting 80% CPU on a web tier. There is no room left while new instances boot.

Check yourself

1. Why target 50% CPU instead of 80%? Answer: New instances take minutes to boot and warm up, and the spare headroom absorbs the spike in the meantime.
2. Which workload suits spot capacity? Answer: Stateless, interruptible work such as batch jobs and CI runners, ideally with checkpoints.

Key takeaways

• Choose a VM family from the resource that runs out first, and prefer the newest generation.
• Spot is 60 to 90% cheaper but can be reclaimed with a two-minute (AWS) or 30-second (GCP) warning.
• An autoscaling group keeps a fleet at a target metric and replaces unhealthy instances.

Further learning

Next, see the three kinds of storage the instances attach to: storage types.