Shipping and specialties
Quantifying feedback
A lightweight taxonomy plus counts and a linked metric turns anecdotes into decisions — and stops you staffing the wrong team.
Raw feedback is not a backlog. Run a lightweight taxonomy so themes can be counted. Then join those counts to behavior. Counting without categorizing staffs the wrong team. Categorizing without counts lets anecdotes pretend to be a referendum.
When you cannot quantify, say so explicitly — "anecdotal from 4 design partners" — so it does not wear a vote costume.
Concept
A theme row is one problem, counted: mentions, accounts, estimated % of actives affected, severity and a linked metric. If the linked metric is blank, you are volunteering for zombie work.
Walking through a false reliability fire
Example tags for a B2B SaaS over one month: activation / onboarding; performance / reliability; pricing confusion; missing integration X; permission / admin friction; "works but hard to find"; feature request (net-new).
| Theme | Mentions (30d) | Accounts | % of actives affected (est.) | Severity | Linked metric |
|---|---|---|---|---|---|
| SMTP setup fails | 42 tickets + 7 beta notes | 31 | ~8% of teams trying schedules | High (blocks aha) | activation rate |
| Want Slack notify | 28 votes + 11 sales | 40 | unknown desire | Low (workaround email) | — |
Support tags 60 tickets "export broken." Analytics shows export success rate 99.2%. Digging in: 55 tickets are "where is export?" (discoverability), 5 are real failures. The roadmap item is not a reliability project — it is navigation and empty-state copy.
Quantify when you can: ticket volume, % of weekly actives hitting an error code, funnel drop between step A and B, revenue on accounts requesting a capability.
Every feedback-sourced ticket should answer four fields before it is "Ready":
- Evidence — count + channels + time window.
- User / job — who and what job, not just the requested widget. See jobs to be done.
- Success signal — what metric or qualitative check would change if we ship.
- Decision — build / research more / won't-do (with rationale).
If field 3 is blank, you are volunteering for zombie work. Won't-do is a first-class outcome of triage; publish it so the same request does not reappear every quarter under a new champion.
Common mistakes
- Counting tickets without reading them. "Export broken" was "where is export?"
- Letting vote boards set priority. Selection bias plus gaming.
- Skipping the success signal. You will never know if the ship worked.
Check yourself
60 tickets say "export broken." Success rate is 99.2%. What is the real theme?
Mostly discoverability (55 of 60). Staff navigation and copy, not a reliability squad.
A ticket is "Ready" with no success signal. What do you do?
Send it back. Without a metric or qualitative check, it is zombie work.
Key takeaways
- Taxonomy + counts + linked metrics turn anecdotes into decisions.
- Join qualitative themes to analytics before you staff a team.
- Four fields on every feedback ticket: evidence, job, success signal, decision.
Further learning
Next, tell people what you did in closing the feedback loop.
Related
Iteration and feedback loops
Turning feedback, support tickets and data into continuous improvement and kill decisions.
Feedback channels
Every channel lies in a different way. Combine support, in-app, sales and behavior — never let volume from one source set the roadmap.
Product analytics
Designing event tracking plans, reading funnels and cohorts, writing the SQL that answers product questions, and avoiding mix-shift illusions. Start here for the map.