Think of a system prompt as the standing order you give a new employee before their first day. You do not explain your company culture every morning. You do it once — clearly, completely — and then you expect them to work within it without being reminded.
A system prompt is exactly this. It is the instruction you give the model before any conversation begins. It sets the context, the role, the constraints, and the tone. Every message the user sends is read in light of it.
Why it matters more than your individual messages
Most people spend their time optimising individual prompts — the specific question they ask. But the system prompt determines the model’s baseline behaviour across everything. A well-written system prompt means you spend less time correcting and clarifying in individual messages.
A model with no system prompt is a generalist with no context about your use case. A model with a good system prompt is a specialist who understands your domain before you say a word.
What to put in a system prompt
The most effective system prompts have four elements:
Role. What is the model? “You are a customer support agent for a developer tools company.” Not “You are a helpful assistant.” Specific roles produce specific behaviour.
Context. What does the model need to know to do its job? Not everything — only what is relevant. “Our users are developers. They value precision over friendliness. Technical accuracy matters more than tone.”
Constraints. What should the model never do? “Do not make up information. If you do not know the answer, say so.” Constraints prevent the most common failure modes.
Format. How should responses look? “Respond in plain text. No bullet points unless specifically requested. Keep responses under 200 words unless the question requires more detail.”
The cost consideration
System prompts are sent with every request. A 1,000-token system prompt sent 100,000 times costs the same as 100,000,000 additional input tokens. Most providers offer prompt caching — where a repeated system prompt is cached and not re-charged after the first call. Enable it. It typically reduces system prompt costs by 90%.
The one thing to do today
Take your current system prompt — or write one if you have not — and read it aloud. If any sentence is vague, it will produce inconsistent behaviour. Tighten every sentence until it means exactly one thing. Then test it against ten different inputs to verify the model behaves as expected.