Metrics and experimentation
AARRR and leading metrics
Pirate metrics map the journey. They do not replace a north star. Prefer metrics that move in weeks over ones that only confirm last quarter.
Dave McClure's AARRR ("pirate metrics") is a map of the customer journey, useful for spotting where a metric tree is broken. It is a funnel sketch, not a substitute for a north star. Companies often over-invest in Acquisition because it is visible, while Activation and Retention determine whether acquisition spend compounds or burns.
Finance metrics (ARR, gross margin) are mostly lagging: they confirm what already happened. A good north star is leading enough that product changes show up in days or weeks, yet lagging enough that it is not pure vanity motion. "Buttons clicked" leads everything and means nothing. "Cash collected this quarter" means everything and arrives too late to steer a sprint.
The big idea
| Stage | Question | Example metrics |
|---|---|---|
| Acquisition | How do users find you? | Visits, installs, CAC by channel |
| Activation | Did they reach an aha moment? | % completing setup, time-to-first-value |
| Retention | Do they come back? | D1/D7/D30 retention, cohort curves |
| Referral | Do they bring others? | Invite rate, k-factor |
| Revenue | Do they pay / expand? | Conversion to paid, ARPU, NRR |
Instrument all five stages; pick one north star that usually sits near Activation × Retention × depth of use.
A weekly rhythm that works at many companies:
- Daily: glance at guardrails and severe regressions (errors, payment failures).
- Weekly: review the north star and its 3–5 inputs with owners; ask which input moved and why.
- Monthly: reconcile the north star against revenue / retention. If they diverge for two months, either monetization is broken or you chose the wrong star.
A highway dashboard is a useful picture: speed is leading, destination arrival is lagging, fuel light is a guardrail. The analogy breaks when you optimize speed by taking a shortcut off a cliff — hence guardrails on the same review.
Walking through it
In interviews ("what's the north star for X?"), structure the answer: customer value → candidate metric → why it leads revenue → 3 inputs → 2 guardrails. Mention one way the metric could be gamed and how you would detect it.
Process for choosing a star: write candidates mapped to customer value moments from jobs to be done; check leading correlation with revenue or retained users over 6–12 months (directionally); decompose into inputs; publish definitions in the analytics tracking plan; review quarterly; change rarely.
Ronny Kohavi's finding that most ideas fail in controlled tests is useful context: picking the right metric matters before you A/B everything.
Common mistakes
- AARRR as religion. Five stage dashboards without one shared star. Teams still argue about features.
- Steering by ARR in a sprint. Too lagging. Pair it with an activation or usage input.
- Acquisition theater. Paid ads are visible; a leaky activation step makes them expensive.
Check yourself
- Where does a north star usually sit on the AARRR map?
Near Activation × Retention × depth of use — the behavior that causes revenue, not the ad click.
- North star up 12% over eight weeks, paid conversion flat. What should the monthly review ask?
Is monetization broken, or did the star's quality drift? Segment the star (e.g. multi-editor share) before celebrating.
Key takeaways
- AARRR maps the journey; it does not replace a strategy-aligned north star.
- Leading enough to steer a sprint, lagging enough to mean value — not button clicks, not only cash collected.
- Review the star weekly against inputs; reconcile to revenue monthly.
Further learning
Ronny Kohavi on why most ideas fail in controlled tests:
Back to the map: product metrics.
Related
Product metrics and the north star
Choosing a north star that reflects customer value, decomposing it into input metrics teams own, and pairing it with guardrails. Start here for the map.
Guardrails and vanity metrics
Pair every primary metric with 2–4 guardrails so you cannot silently destroy trust. Vanity metrics move easily and do not change what you build next week.
Retention and engagement
Retention curves, cohort analysis, DAU/MAU and why retention is the foundation of growth. Start here for the map.