What is a system prompt?
In Lesson 01, you made a single call: prompt β completion. Today you'll extend that call to include two new parameters: system and temperature.
A system prompt is an invisible instruction that sets the role, tone, and constraints for the entire conversation. The user's prompt (the one they see) never changes. The system prompt changes how the model interprets and responds to it.
User: "What's the refund policy?"
Output: 50% response, 50% tangent about company history.
System: "You are a support agent. Be concise (under 60 words). Point to policy page."
Output: Focused, policy-aware, exactly on-brand.
System vs user prompt β when to use which
System prompt: Set once per conversation. Defines the role, guardrails, output format, and tone. Example: "You are a financial analyst. Always cite data sources."
User prompt: Changes per turn. The actual question or task. Example: "Summarize Q1 earnings for SouthBank India."
Think of it this way: the system prompt is the job description. The user prompt is the task assigned each day.
System prompt stays constant: Once set, it applies to all turns in the conversation. Good for maintaining voice and constraints across multiple exchanges.
User prompt changes: Each new question is a new user prompt. The model remembers the system prompt and applies it to the new question.
Temperature: control randomness (0 = fixed, 1+ = creative)
Temperature controls how "creative" or "random" the model's output is. It ranges from 0 to 2, though 0β1.5 covers most use cases.
Technically: Temperature scales the "softmax" probabilities of the next token. At T=0, the model always picks the token with the highest probability. At T=1, the probabilities are used "as-is". At T>1, low-probability tokens become more likely, creating wild variation.
Extend the chat() helper with system and temperature
In Lesson 01, you called chat(prompt, provider). Now add system= and temperature=.
system parameter now sets the model's role. The temperature parameter controls randomness. Both are passed directly to Ollama and Gemini APIs. Same interface, different providers underneath.
Worked example 1: Aisha's customer support agent (Jumia Kenya)
Aisha works at Jumia Kenya in Nairobi. She wants to automate first-response customer support: quick, polite, on-policy. She'll use a system prompt to enforce constraints and temperature=0 for consistency.
Code Block 1: The support agent (temperature=0)
Code Block 2: Same agent, temperature=0.7 (more natural)
Now let's see what happens when we allow some variation. The answer is still on-brand, but with more personality.
Code Block 3: temperature=1.2 (creative, risky)
At high temperature, the model explores less-likely responses. Watch what happens:
Worked example 2: SouthBank India earnings summary bot
Arjun works in financial analysis at SouthBank India's Mumbai office. He builds a bot that summarizes earnings data in exactly 3 bullets. System prompt enforces format. Temperature=0 ensures consistency.
Few-shot prompting: teach by example
Sometimes words aren't enough. You can embed examples inside the system prompt to teach the model the exact format and tone you want.
Anti-patterns: what breaks and how to avoid it
System prompts are powerful but have limits. Here are three common mistakes:
Anti-pattern 1: Putting secrets in the system prompt
Don't: Put API keys, passwords, or private data in the system prompt. If you send a request to a cloud provider (Gemini, ChatGPT API), the system prompt travels over the internet.
Do: Keep secrets in environment variables or backend code. Pass only public information to the model.
Anti-pattern 2: Contradicting instructions across turns
Don't: Set a system prompt saying "Always be brief" but then the user says "Give me 500 words." The model will be confused and sometimes ignore the system prompt.
Do: Make system prompts defensive. Example: "Keep replies under 100 words unless explicitly asked for more." This way, user input can override without contradiction.
Anti-pattern 3: Over-constraining with temperature=0
Don't: Use temperature=0 for creative tasks (content generation, brainstorming). The model will repeat the same phrases and feel robotic.
Do: Match temperature to task: 0 for deterministic (support, data extraction), 0.7β1.0 for balanced (chatbot, summarization), 1.0+ for creative (ideation, writing).
Write a system prompt that ensures the bot:
- Greets users in English and Swahili
- Keeps replies under 80 words
- Points transaction issues to https://m-pesa.co.ke/help
- Stays conversational but professional
Test it with: "My transaction failed. I was sending β±500 to my mum."
Start with: "You are an M-Pesa support agent in Kenya..." Then add each constraint as a separate line. End with an example reply tone.
Test your knowledge
You've shaped your first agent.
Now build ones that solve real problems.
Join the Best AI Program: The AI-Era Agentic Engineer β 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Agentic Engineer β