Observability and reliability
Blameless postmortems
Learning from failure with timelines, contributing factors and action items that actually get done — without turning people into the root cause.
An incident ends when users are safe again. The postmortem starts then. It is the written record of what happened, how bad it was, what contributed, and what will change. Without it, the same page wakes someone else in six months, and the organization learns only fear.
Response stops the bleeding. The postmortem remakes the system. The two pages sit side by side: incident response covers the live work, and this topic covers the write-up that follows.
Incidents repeat when the learning stays in one person's head or one team's folder. A good postmortem has a clear summary, a timeline built from machine data, several contributing factors, and action items that someone owns. Then the next failure of the same kind is smaller, or it never happens.
Concepts in reading order
| Step | Concept | What it answers |
|---|---|---|
| 1 | Blameless culture | Why ask which conditions failed, not who? |
| 2 | Summary and timeline | How do you write the impact and the record? |
| 3 | Contributing factors | How do you list several causes without blaming one? |
| 4 | Action items | How do you make sure the fixes get done? |
| 5 | Sharing learnings | How do lessons reach other teams? |
Read step 1 first. It explains the rules the later steps use.
How a postmortem flows
Common mistakes
- Writing the document and never reviewing the actions. Review them weekly until they are closed.
- Blameless wording that still shames. "An unfortunate oversight by the team" is still blame.
- Skipping impact numbers. Leadership cannot prioritize reliability work without them.
- Writing a novel for every small incident. Match the depth to the severity.
In practice
As an SDE2 you will write postmortems for incidents you caused, incidents you mitigated and incidents you only watched. Your job in the document is clarity and useful follow-ups, not self-defense. In interviews, be ready to walk through a failure you owned, using the template. Interviewers listen for whether you fix systems or blame people.
Key takeaways
- Blameless means honest data. Accountability means real change. Keep both.
- Use the template: summary, impact, timeline, detection, contributing factors, what went well, action items.
- Name several contributing factors, not one root cause that lands on a person.
- Give every action item one owner, a priority and a date, and track it to done.
- Share widely. Incident libraries and reviews multiply the value of each outage.
Further learning
Google's SRE book chapter is the canonical framing of blameless culture.
Postmortem Culture: Learning from Failure (Google SRE book)Blameless postmortems in SRE culture. Incident study that focus on root cause analysis and preventive actions, for culture of continuous improvement.sre.googleThe SRE book's example postmortem shows the template filled in with numbers and several causes.
Example Postmortem (Google SRE book)This practical site reliability incident postmortem example outlines blameless analysis, action items and a detailed timeline for effective outage resolution.sre.googleCloudflare's blog keeps a library of public postmortems. The July 2019 WAF outage is a good one to read for process-aware analysis.
Cloudflare blog: post-mortemsAll posts tagged with Post MortemCloudflare Blog
Related
Incident response
Run an incident with clear roles, severity and communication. Mitigate first, debug second, then hand off to the postmortem. Start here for the map of incident pages.
Owning your code in production
On-call, alerts, runbooks, and the loop that makes you trustworthy — and better at building systems that survive users.
Written and verbal communication
Lead with the point, write for skimmers, and adjust depth for engineers, PMs and executives.