GitHub went down yesterday and we found out the way most teams find out: something that normally takes one click quietly stopped happening, and we spent a while working out whether the problem was ours.
That gap — the minutes between "something is odd" and "it is not us" — is the part worth talking about. Everything after it is just waiting.
Because the interesting question during an outage is not when it will be fixed. You have no say in that, and refreshing a status page is not work. The question is what your team can still do.
And that is a question about who knows what.
An outage is an audit nobody scheduled
Who can deploy by hand? Who knows what the automation was actually doing, as opposed to what its job was called? Who can tell a genuinely broken build from a runner that never started?
In most teams, that is one person. And that person is the entire reason the outage felt survivable.
Worth saying plainly: that is a single point of failure with a salary, an inbox, and annual leave.
Our own tell, from the same week: a workflow that "fails" in a few seconds having run zero steps does not mean the code is broken. It means the runner never started — in our case because the Actions budget was gone.
Read that as a red build and you send someone debugging a thing that was never wrong. The misread is expensive precisely because it is so plausible: red cross in the UI, "failed" in the notification, and every instinct says look at the diff.
Why your postmortem will miss this
The standard template asks what broke, when, and what we will change. Fine questions. They produce a document that is mostly useful to people who were already in the room.
What it almost never captures is the diagnostic path — the three things that looked like the cause and were not, and how each was ruled out.
That is the expensive knowledge. Reconstructing a timeline from logs is easy months later. Reconstructing why a competent person spent forty minutes on the wrong hypothesis is impossible, because nobody wrote down that they had it.
So the notes record the route once it was known, and lose the search that found it.
Two questions worth adding
After the next incident, ask these alongside the timeline:
What looked broken and was not? And what was the tell that separated them?
What did we do by hand that is normally automated? Write the commands down — next time the person who knows them may be on a plane.
Then, once a quarter, hand the notes to someone who was not there and ask them to follow along. If they cannot, you have a souvenir, not a runbook. That test takes an afternoon and it is the only one that actually tells you whether the knowledge left the building with a person or stayed.
Google's SRE practice has argued something close to this for years, and this is a good short version of the culture it takes:
The incident itself is on GitHub's status page, and it is unremarkable. Ours always are, right up until we cannot answer a simple question about our own systems.
One email when we publish. Research, product decisions, and what teams report back.