What is a system prompt?

In Lesson 01, you made a single call: prompt β†’ completion. Today you'll extend that call to include two new parameters: system and temperature.

A system prompt is an invisible instruction that sets the role, tone, and constraints for the entire conversation. The user's prompt (the one they see) never changes. The system prompt changes how the model interprets and responds to it.

Without system prompt
Question only

User: "What's the refund policy?"

Output: 50% response, 50% tangent about company history.

With system prompt
Role + constraints

System: "You are a support agent. Be concise (under 60 words). Point to policy page."

Output: Focused, policy-aware, exactly on-brand.

Key insight:
The system prompt is not a constraint on the model's knowledge. It's a constraint on the model's behavior and output format. The model already knows about refund policies. The system prompt just says "emphasize the policy page and stay under 60 words."

System vs user prompt β€” when to use which

System prompt: Set once per conversation. Defines the role, guardrails, output format, and tone. Example: "You are a financial analyst. Always cite data sources."

User prompt: Changes per turn. The actual question or task. Example: "Summarize Q1 earnings for SouthBank India."

Think of it this way: the system prompt is the job description. The user prompt is the task assigned each day.

System prompt stays constant: Once set, it applies to all turns in the conversation. Good for maintaining voice and constraints across multiple exchanges.

User prompt changes: Each new question is a new user prompt. The model remembers the system prompt and applies it to the new question.

Temperature: control randomness (0 = fixed, 1+ = creative)

Temperature controls how "creative" or "random" the model's output is. It ranges from 0 to 2, though 0–1.5 covers most use cases.

Deterministic
0
Same prompt = same output, every time. Model picks the most likely token.
Use for: Customer support scripts, data extraction, consistency-critical tasks.
Balanced
0.7
Some variation, but still focused. Good default for most tasks.
Use for: Chatbots, summarization, general Q&A where human touch helps.
Creative
1.0+
High variation. Model explores less-likely next tokens. Useful for brainstorming.
Use for: Content generation, creative writing, brainstorming sessions.

Technically: Temperature scales the "softmax" probabilities of the next token. At T=0, the model always picks the token with the highest probability. At T=1, the probabilities are used "as-is". At T>1, low-probability tokens become more likely, creating wild variation.

Extend the chat() helper with system and temperature

In Lesson 01, you called chat(prompt, provider). Now add system= and temperature=.

llm.py (updated) Extended chat() helper with system & temperature
What changed:
The system parameter now sets the model's role. The temperature parameter controls randomness. Both are passed directly to Ollama and Gemini APIs. Same interface, different providers underneath.

Worked example 1: Aisha's customer support agent (Jumia Kenya)

Aisha works at Jumia Kenya in Nairobi. She wants to automate first-response customer support: quick, polite, on-policy. She'll use a system prompt to enforce constraints and temperature=0 for consistency.

Code Block 1: The support agent (temperature=0)

support_agent.py Deterministic support replies with system prompt
Expected output (temp=0.0)
Jambo! I'm sorry to hear your order hasn't arrived yet. Orders usually take 3-5 business days in Nairobi. Can you check your email for tracking details? If it's delayed, reach out to support@jumia.ke and we'll investigate right away. Thank you for your patience!

Code Block 2: Same agent, temperature=0.7 (more natural)

Now let's see what happens when we allow some variation. The answer is still on-brand, but with more personality.

support_agent_varied.py Same system prompt, different temperature
Expected output (temp=0.7)
Habari! Thanks for reaching out. Most orders arrive within 3-5 days. Check your email for a tracking linkβ€”it'll show exactly where your package is. If something seems off, our team at support@jumia.ke is ready to help. We appreciate your patience!

Code Block 3: temperature=1.2 (creative, risky)

At high temperature, the model explores less-likely responses. Watch what happens:

support_agent_creative.py High temperature = unpredictable (usually not good for support)
Expected output (temp=1.2)
Habari, jambo, and welcome! Ah, the anticipation of waiting for your Jumia treasure. Fear not, dear customerβ€”our magical logistics team works around the clock. Do check your email for mystical tracking updates. If the stars aren't aligned, contact support@jumia.ke for celestial intervention!
The lesson:
Same system prompt, three different outputs. temp=0 is predictable (good for production). temp=0.7 feels human (good for chat). temp=1.2 is creative but unreliable (bad for support, great for brainstorming). For customer support, Aisha should use temp=0 or 0.5 to stay on-brand.

Worked example 2: SouthBank India earnings summary bot

Arjun works in financial analysis at SouthBank India's Mumbai office. He builds a bot that summarizes earnings data in exactly 3 bullets. System prompt enforces format. Temperature=0 ensures consistency.

earnings_bot.py Financial bot with strict format enforcement
Expected output
- Revenue reached β‚Ή45,200 Cr in Q1 2026, growing 18% YoY. Net profit of β‚Ή8,450 Cr reflects strong operational performance. - Profit margin improved with stable NPA ratio of 2.1%. Digital transactions surged 45% YoY, indicating successful digital transformation. - Regulatory capital ratio of 16.8% comfortably exceeds minimum requirements, strengthening the bank's financial position and risk management.

Few-shot prompting: teach by example

Sometimes words aren't enough. You can embed examples inside the system prompt to teach the model the exact format and tone you want.

few_shot_example.py System prompt with 2 examples embedded
Expected output
Apple AirPods Pro (2nd Gen) | $249 | Premium noise cancellation and personalized audio in one wireless earbud pair
Few-shot power:
Instead of just saying "summarize concisely," you show 2 real examples. The model learns the exact format, tone, and length from those examples. This is more reliable than describing the format in words.

Anti-patterns: what breaks and how to avoid it

System prompts are powerful but have limits. Here are three common mistakes:

Anti-pattern 1: Putting secrets in the system prompt

Don't: Put API keys, passwords, or private data in the system prompt. If you send a request to a cloud provider (Gemini, ChatGPT API), the system prompt travels over the internet.

Do: Keep secrets in environment variables or backend code. Pass only public information to the model.

Anti-pattern 2: Contradicting instructions across turns

Don't: Set a system prompt saying "Always be brief" but then the user says "Give me 500 words." The model will be confused and sometimes ignore the system prompt.

Do: Make system prompts defensive. Example: "Keep replies under 100 words unless explicitly asked for more." This way, user input can override without contradiction.

Anti-pattern 3: Over-constraining with temperature=0

Don't: Use temperature=0 for creative tasks (content generation, brainstorming). The model will repeat the same phrases and feel robotic.

Do: Match temperature to task: 0 for deterministic (support, data extraction), 0.7–1.0 for balanced (chatbot, summarization), 1.0+ for creative (ideation, writing).

Challenge: Build your own agent
Task: Create a system prompt for an M-Pesa customer support bot (Kenya).

Write a system prompt that ensures the bot:

  • Greets users in English and Swahili
  • Keeps replies under 80 words
  • Points transaction issues to https://m-pesa.co.ke/help
  • Stays conversational but professional

Test it with: "My transaction failed. I was sending β‚±500 to my mum."

Start with: "You are an M-Pesa support agent in Kenya..." Then add each constraint as a separate line. End with an example reply tone.

Test your knowledge

Question 1 of 5
What is the difference between a system prompt and a user prompt?
Question 2 of 5
At what temperature does the model always give the same output for the same input?
Question 3 of 5
You're building a customer support bot. Which temperature is best?
Question 4 of 5
What is few-shot prompting?
Question 5 of 5
Why should you NOT put API keys in the system prompt?
0
/5
Excellent! You understand system prompts and temperature.
Africa & India's Best AI Academy
πŸ€– AI-first πŸŽ“ Placement Assistance

You've shaped your first agent.
Now build ones that solve real problems.

Join the Best AI Program: The AI-Era Agentic Engineer β€” 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Agentic Engineer β†’