The problem: each call forgets everything

In Lesson 01, you called the LLM once and got one answer. But real applications need conversations. A user asks a question, the bot answers. The user asks a follow-up based on the answer. The bot should remember what came before.

If you call the LLM function the same way twice, it has no memory of turn 1. It doesn't know the user's account number from 30 seconds ago. It can't refer to the previous exchange.

Turn 1: "What's my balance?"
→
Model: "₹15,000"

Turn 2: "Can I withdraw ₹5000?"
→
Model: "I don't know your balance." ✗ Lost context!

This is the gap we close in this lesson. Multi-turn conversation requires state—a list of messages that grows with each turn.

How chat APIs actually work: the messages array

Every chat endpoint (Ollama, Gemini, Claude) uses the same pattern: a messages array with roles. Each message has three fields:

role
"system", "user", or "assistant"
content
The actual text of the message
function
(Optional, for tool-using agents later)

A conversation with 3 turns looks like this:

messages_structure.py The messages array pattern
Key insight:
You don't send just the current turn. You send the entire history each time. The model reads all previous messages and understands context. This is why bigger context windows matter—if history gets too long, you hit token limits.

Building conversation state: a ChatSession class

The practical way to build multi-turn chat is a class that maintains the message history. Here's a minimal ChatSession:

chat_session.py Minimal ChatSession that maintains history
Expected output
Your balance is ₹17,500. Yes, you can transfer ₹5000 from account SB-2026-4521. Your remaining balance will be ₹12,500.
What happened:
Turn 2's response mentions "account SB-2026-4521" because the model saw it in turn 1. The history grows automatically. No manual state tracking needed.

The context window problem: token limits

Every model has a maximum context window—the total number of tokens it can read. Common limits:

Model Context Window Real use
Ollama (Mistral 7B) 8,000 tokens ~5 pages of text or 20-turn conversation
Ollama (Llama 70B) 4,096 tokens ~2 pages or 8-turn conversation
Gemini 2.0 1,000,000 tokens Entire books + 500-turn conversation

If your conversation history exceeds the context window, the model returns an error. For a chatbot that runs for hours, you need a strategy to manage this.

Three strategies: keep, summarize, hybrid

Strategy 1: Sliding Window (keep last N turns)

Discard old messages, keep only the most recent N turns. Simple but loses long-term context.

sliding_window.py Keep only last 5 turns to stay under token limit

Strategy 2: Summary Compression (summarize old turns)

Periodically call the LLM to summarize the conversation history into bullet points. Keep that summary and discard old messages.

summary_compression.py Summarize history every 10 turns

Strategy 3: Hybrid (use both)

In production, combine sliding window (keep recent turns) with summary compression (preserve key facts from early conversation). This balances recency with long-term context.

Worked example: Jumia Kenya customer support bot

Arjun wants to build a bot for Jumia Kenya customer support. In turn 1, the customer gives their order number. In turn 2, they ask a question. In turn 3, they ask another question—and the bot should still know the order number without the customer repeating it.

🛒
Real scenario
Jumia Kenya: order tracking across 3 turns

Turn 1: "My order is JK-2026-98765."

Turn 2: "Where is my package?"

Turn 3: "Can I get a refund?"

Without multi-turn chat, turn 2 and 3 forget the order number. With our ChatSession, the bot remembers across all turns.

jumia_bot.py Jumia Kenya support bot that maintains order context
Expected output (simulated)
Hi! I'm sorry to hear that. Let me help with order JK-2026-98765. What's the issue? I understand. Let me check order JK-2026-98765. This shows a delivery delay. Let me escalate this for you. For order JK-2026-98765, yes—since it's overdue, I can process a refund to your M-Pesa. This takes 2-4 hours.

Saving and loading sessions to JSON

Users close apps. Bots need to pause and resume. Save the message history to JSON, then reload it later.

persistence.py Save and load conversation sessions

Next: tools + conversation = agents

This is the foundation. In Lesson 05, we give the chat session access to tools. Instead of just talking, the bot can call functions—fetch real order data from Jumia's API, check M-Pesa balance, initiate refunds. A ChatSession that can do things is an agent.

🎯 Mini Challenge: Build a SouthBank banking bot

Task: Create a ChatSession for a SouthBank India customer support bot. Run 4 turns:

1. Customer: "My account is SB-5001-Hyderabad."

2. Customer: "What's my recent transaction history?"

3. Customer: "Can I dispute one of those charges?"

4. Customer: "How long will that take?"

Verify that turn 4's response mentions the account number or disputes from earlier turns.

Hint: Use ChatSession from chat_session.py. Create a system prompt for a SouthBank agent, then call session.chat() four times. Print each response.

How AI systems use this

Every chatbot and conversational AI (Siri, ChatGPT, customer service bots) uses the messages array pattern. Larger language models (Claude, GPT-4) have larger context windows, so they can remember longer conversations. But the core pattern is identical: accumulate messages, send full history, extract new message from response, repeat. This is how LLMs "understand" conversation—not through special memory, but by reading the full chat history each time.

Conversation Design State Management Context Windows Agents

Test your knowledge

Question 1 of 5
In a multi-turn chat API, what do you send to the model on turn 2?
Question 2 of 5
What does the "role" field in a message object specify?
Question 3 of 5
Why is context window size important for chat systems?
Question 4 of 5
Which strategy keeps long-term context best when conversation history is huge?
Question 5 of 5
How do you make a chat session survive app restarts?
0
/5
Excellent! You understand multi-turn chat and conversation memory.
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

You've built stateful chat.
Now add tools and launch real agents.

Join the Best AI Program: The AI-Era Agentic Engineer — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Agentic Engineer →