Security, cost and GPUs
Cloud cost and FinOps
Where cloud money goes, the levers that usually cut a bill, and how engineers own unit cost without waiting for a finance spreadsheet. Start here for the map of cost pages.
Cloud bills surprise teams that scaled features faster than they watched unit cost. A service at $40k a month can reach $180k after one unindexed query, a chatty cross-zone pattern and a logging pipeline that ingests every debug line. FinOps is the practice of making cloud spend visible, assigning it to owners, and optimizing it continuously. Engineers are part of that loop, not only finance.
The cloud primitives page covers what you buy. The gross margin page covers why cost matters to the business.
The core idea
Open the cost explorer by service, account and tag. Do not start from the invoice total. A total tells you the bill went up. Service and tag tell you where, and whose.
Most bills concentrate in a few categories. Compute, databases, storage, data transfer and observability cover most of the spend. Each category has its own levers.
Read in order
| Step | Concept | What it answers |
|---|---|---|
| 1 | Where a cloud bill goes | Which categories dominate, and what does a real reduction look like? |
| 2 | Pricing models and commitments | When do on-demand, committed and spot capacity each make sense? |
| 3 | Hidden network and logging costs | Why do NAT, cross-zone traffic and logs cost so much? |
| 4 | Tagging and unit cost | How do you assign spend to owners, and tie it to product metrics? |
Architecture beats discounts
Discount programs save tens of percent. Architecture can save multiples when traffic patterns are wrong. Chatty services across zones, debug logging on every request, and multi-region copies you do not need all multiply the bill. Fix the shape first, then buy the discount.
A 40% commitment on an architecture that wastes half its cycles still leaves a large part of the bill on the floor.
Cloud or your own hardware
Some companies move large, steady workloads off the public cloud and onto owned hardware. The savings can be large, but they need an ops team that can run the hardware. Most companies should exhaust rightsizing, commitments and architecture first. Repatriation is a strategy for a mature cost center, not a default.
Common mistakes across the topic
- Starting with discounts. Shape the workload first.
- Debug logs and full traces everywhere. Sample by default, and raise detail on error paths.
- Multi-region by default. It doubles many line items. Use it for latency or hard disaster recovery needs.
- Forgetting orphans. Unattached disks, old load balancers and snapshots of deleted volumes all keep billing.
- Keeping launch leftovers. Idle replicas and debug logging become permanent after a launch.
Key takeaways
- Explore the bill by service and tag, not by total.
- Mix on-demand, commitments and spot on purpose. Spot needs tolerant workloads.
- Hidden multipliers are NAT, cross-zone traffic and log ingest.
- Tags and unit cost make spend a product signal.
- Optimize the cloud before you consider repatriating.
Further learning
The FinOps Foundation's introduction defines the practice and the vocabulary you share with finance.
FinOps Foundation: What is FinOps?finops.orga16z's essay frames when cloud rent dominates SaaS costs, and when owning hardware enters the conversation.
The Cost of Cloud, a Trillion Dollar Paradox (a16z)The pressure the cloud puts on margins can start to outweigh the benefits you scale and growth slows. Understand how much market cap is being suppressed by the cloud to help inform the decision-making framework on man…Andreessen Horowitz
A grounded talk on moving from one-off optimization to engineering culture at scale:
Related
Gross margin and COGS, including AI
What belongs in COGS vs opex, why classic SaaS runs ~75–80% gross margin, why AI products often sit at 50–60%, and the levers that move the number.
Cloud primitives on AWS and GCP
Regions, zones, compute, storage, managed databases and messaging, how the services map between AWS and GCP, and how to decide what to run yourself.
The economics of inference
Turn $/GPU-hour into $/million tokens with utilization and throughput, then decide API vs self-host and price products with real margins.