Linux and networking
Routes, internet gateways and NAT
A route table picks the next hop for each packet. Public subnets reach the internet through a gateway. Private subnets get outbound access through NAT, and endpoints skip NAT for cloud APIs.
A packet leaving a subnet is sent to the next hop chosen by the subnet's route table. The table is a list of destination prefixes and targets. The rule is simple: the most specific matching route wins. Public and private subnets differ only in which routes they hold.
The big idea
Think of a route table as a set of signposts at a junction. A signpost that says "Manchester" beats one that says "England", because it is more specific. A packet for 10.16.0.0/20 takes the local route. A packet for anywhere else falls through to the default route, 0.0.0.0/0.
Walking through a request
Here are the routes for a private app subnet.
| Destination | Target | Typical use |
|---|---|---|
VPC CIDR (10.16.0.0/16) | local | Traffic inside the VPC |
| S3 prefix list | Gateway endpoint | Private path to S3, no NAT |
0.0.0.0/0 | NAT gateway | Everything else, such as package downloads |
Trace a request to S3 from an app instance. The destination matches the S3 prefix, which is more specific than 0.0.0.0/0, so the packet goes to the endpoint and never reaches NAT. Trace a request to a third-party API. Only the default route matches, so it goes to NAT. NAT then sends it out through the internet gateway.
A public subnet has a route 0.0.0.0/0 to an internet gateway (IGW). Instances in it can have public IPs. An IGW is scaled for you and has no hourly charge, though you pay for data.
A private subnet has no route to the IGW. To reach the internet, outbound traffic goes through a NAT gateway (AWS) or Cloud NAT (GCP). The NAT gateway sits in a public subnet, so private instances can start connections out, but nothing can start one in.
Cost and failure trade-off
Trade-off
One NAT gateway per zone avoids cross-zone data charges, and it survives the loss of a zone for egress. But three NAT gateways triple the hourly and per-GB charges. One NAT gateway is cheaper and is a single point of failure. For production, use one per zone. For small or non-critical environments, one is fine.
Use VPC endpoints for AWS services such as S3 and DynamoDB. A gateway endpoint is a route-table entry with no hourly charge. Interface endpoints (PrivateLink) add an ENI in your subnet and a small hourly charge. Both cut NAT data charges.
Common mistakes
- Putting instances in a private subnet with no NAT. Then
apt updateand package installs silently hang. - Routing S3 traffic through NAT. It costs per GB, when an endpoint would be free.
- Creating NAT in a private subnet. It has no route to the internet gateway, so it cannot serve anyone.
Check yourself
- An app in a private subnet calls S3, and the subnet has both
0.0.0.0/0to NAT and an S3 gateway endpoint. Which route is used? Answer: The S3 endpoint, because its prefix is more specific than the default route. - Why does the NAT gateway live in a public subnet? Answer: It needs a route to the internet gateway. Private instances send their default traffic to it.
Key takeaways
- Routing picks the most specific matching route, and the default route is
0.0.0.0/0. - Public subnets route to an internet gateway. Private subnets reach out through NAT.
- NAT per zone is the production default, and endpoints avoid NAT for cloud APIs.
Further learning
Next, see how traffic is allowed or denied once it has a route: security groups and NACLs.
Related
Cloud networking and VPCs
VPCs, subnets, routing, NAT, security groups and private connectivity in AWS and GCP, plus a three-AZ reference architecture and how to debug when A cannot reach B.
Load balancers, DNS and the edge in the cloud
Configuring the traffic layer, from DNS and cloud load balancers through TLS, WAF and CDN to a regional traffic shift. Start here for the map of edge pages.
Cloud cost and FinOps
Where cloud money goes, the levers that usually cut a bill, and how engineers own unit cost without waiting for a finance spreadsheet. Start here for the map of cost pages.