Metrics and experimentation
Product analytics
Designing event tracking plans, reading funnels and cohorts, writing the SQL that answers product questions, and avoiding mix-shift illusions. Start here for the map.
Dashboards do not create insight. Events do — if they are named consistently, attached to a stable identity and queried with the right population in the denominator. Product analytics is the craft of turning user behavior into trustworthy answers: where the funnel breaks, which cohort retains, whether a segment is healing or the mix just shifted.
This is the instrumentation and query layer under your north star and input metrics. Without it, OKRs are theater. Engineers own event correctness the way they own API contracts. A thin tracking plan in PostHog beats a perfect warehouse schema six months late.
Concepts in this section
Read them in order. Each one builds on the one before.
| Step | Concept | What it answers |
|---|---|---|
| 1 | Event tracking plans | What do we log, under whose identity, and how do we know it is not broken? |
| 2 | Product funnels | Where does conversion drop, and which step is the cheaper lever? |
| 3 | Simpson's paradox | Did the rate change inside segments, or did the mix of segments change? |
| 4 | Analytics SQL | What query shapes and debug checklist answer the product question? |
A cohort is a set of users who share a starting event in a time bucket — everyone who signed up the week of Jan 6. A retention table follows that fixed group forward (D1 / D7 / D30) with the denominator locked to cohort size. Calendar "actives this week / ever signed up" is a different number. Deep dive: retention.
Time windows change funnel answers. A 1-day signup→activate window measures onboarding UX. A 30-day window mixes onboarding with email drips. Pick the window that matches the question, then hold it fixed. Critical money steps should fire server-side; compare checkout_paid client vs server weekly.
When a dashboard looks wrong, check definition, deploy, identity merge, mix, bots and timezone before rewriting SQL. When a PM asks "what's our activation rate?", answer with the definition first ("% of signups who create a project within 7 days"), then the number. Take any Slack claim ("retention is up") and ask for cohort, window and segment.
Common mistakes
- UI-chrome event names and late identity merge. The funnel then lies.
- Celebrating aggregates after a campaign changed mix — Simpson's paradox is common in growth data.
- Open-ended funnels for recent cohorts, or a 30-day window judging a day-0 onboarding change.
- Tool delay. Bad events in Amplitude are still bad events. A thin tracking plan beats a perfect warehouse six months late.
Key takeaways
- Events need stable names, useful properties and early identity merge.
- Funnels multiply step rates; size opportunities with arithmetic before debating features.
- Segment before celebrating — mix shift can move the blended rate while every segment worsens.
- Learn a few SQL shapes; debug definition, deploy, identity and mix before rewriting the query.
Further learning
PostHog documentation: Product analyticsProduct analytics helps you understand how people use your product. Use the events you capture to explore adoption, conversion, and retention with…posthog.com
Mode SQL TutorialLearn to answer questions with data using SQL, no prior coding required, guiding you through foundational to advanced SQL skills.ThoughtSpot
Simpson's paradox and segmenting data (Mixpanel)Avoid misleading insights! Learn how Simpson’s paradox affects data analysis and why segmenting data is crucial for accurate businessMixpanelRelated
Product metrics and the north star
Choosing a north star that reflects customer value, decomposing it into input metrics teams own, and pairing it with guardrails. Start here for the map.
Retention and engagement
Retention curves, cohort analysis, DAU/MAU and why retention is the foundation of growth. Start here for the map.
How a SQL query executes
How a database plans and runs a query, which access paths and join algorithms it chooses between, and how to read EXPLAIN ANALYZE when a query is slow. Start here for query performance.