Gross margin levers

Once COGS is classified correctly, the work is moving it. For AI and cloud products the levers are engineering, pricing and product policy — not a finance spreadsheet alone.

The levers

1. Model routing — easy prompts to small/cheap models; frontier models for hard tasks. A 5× cost difference between tiers is common.
2. Caching / prefix reuse — repeated system prompts and RAG contexts should not be paid for twice. See prefix caching and LLM caching.
3. Pricing — usage add-ons, credit bundles, overage fees, higher list for unlimited. Flat $20 with uncapped GPT-class usage is a charity.
4. Committed-use / reserved capacity — cloud and model providers discount 20–50%+ for commits. Harmful if you over-commit; powerful if utilization is predictable.
5. Self-hosting — trade COGS for capex/opex and eng complexity when volume justifies it.
6. Product limits — rate limits, max context, cheaper defaults. Product policy is a margin tool.
7. FinOps hygiene — idle GPUs, over-provisioned SQL, orphaned disks.

Walking through a non-AI cut

Non-AI SaaS at $2M ARR (~$167k revenue/month):

Gross margin = (167,000 − 42,500) / 167,000 ≈ 74.6%.

Cut observability waste by $2k and move one support FTE to a cheaper region (−$5k): COGS → $35.5k → GM ≈ 78.7%. On $2M ARR that is about $82k / year of gross profit — before you touch pricing.

a16z's Cost of Cloud essay argued that many public software companies would be worth more if they owned more of their infrastructure, because cloud COGS silently taxes margin and cash. Whether you repatriate or not, treat cloud as a margin line.

Valuation is a margin event

Two companies, each at $20M ARR, growing 50% YoY. Company A GM 80% → gross profit $16M. Company B GM 55% → $11M. Both spend $14M on opex:

A operating income ≈ 16 − 14 = +$2M
B operating income ≈ 11 − 14 = −$3M

Same growth story. A is already operating-profitable on this sketch. B's fundraise is a margin conversation first. Shipping a feature that permanently adds 5 points of COGS is a valuation event, not only an eng event.

Track GM by product/SKU and by customer tier. The AI add-on may be 40% while the core seat is 85%. Average GM then becomes a vanity metric.

Common mistakes

• Optimizing cloud and hoping the average moves while one SKU is 40% GM.
• Over-committing reserved capacity against a usage forecast you do not have.
• Cutting quality to save tokens without measuring whether conversion or retention falls.

Check yourself

1. A 5-point COGS increase on $50M ARR. How much gross profit disappears? $2.5M per year.
2. Why track GM by SKU, not only company-wide? The AI add-on can be 40% while the core seat is 85%. The average hides the slice to reprice.

Key takeaways

• Routing, caching, pricing, commits, self-hosting, caps and FinOps all move margin.
• Do not pick only one of raise-price / cut-COGS / cap-usage.
• A 5-point COGS change is a valuation event.
• Measure GM by SKU and by tenant, not only the company average.

Further learning

The Cost of Cloud, a Trillion Dollar Paradox (a16z)The pressure the cloud puts on margins can start to outweigh the benefits you scale and growth slows. Understand how much market cap is being suppressed by the cloud to help inform the decision-making framework on man…Andreessen Horowitz

Back to the map: gross margin.