The goal: conversational filing chat

Over the last three modules, you learned embeddings, vector databases, and multi-turn chat. Now you'll tie them together into a real-world document Q&A system.

In this lesson, you'll build a CLI bot that works like this:

$ python filing_chat.py southbank_10q.pdf
Loading filing: southbank_10q.pdf
Embedding and indexing... done.

You: What is SouthBank's revenue for Q1 2024?
Bot: SouthBank reported Q1 2024 revenue of โ‚น8,450 crore, a 12%
year-over-year increase. [Page 3, 'Financial Results']

You: How many branches do they operate?
Bot: SouthBank operates 2,847 branches across India. [Page 7,
'Branch Network']

You: Compare that to the previous year.
Bot: The filing shows 2,731 branches last year, so they added
116 branches in Q1. [Page 6, 'Year-over-year comparison']

The bot remembers every question you ask, retrieves relevant chunks from the PDF, and cites the page number so you can verify.

The 7-step RAG pipeline: parse โ†’ chunk โ†’ embed โ†’ store โ†’ retrieve โ†’ prompt โ†’ answer

Every RAG system follows the same flow: turn a document into chunks, embed them, store them in a vector database, retrieve the closest matches to a query, and feed them to an LLM.

1. Parse PDF
โ†’
2. Chunk text
โ†’
3. Embed chunks
โ†’
4. Store in DB
5. Retrieve on query
โ†’
6. Format context
โ†’
7. LLM + citations
๐Ÿ“Š
Real scenario
Priya's investment research (Mumbai)

Priya is an analyst at a Mumbai investment firm. Every day, she reads 5-10 company 10-Qs and answers stakeholder questions: "What is HDFC's Q2 NIM?" "How many RELIANCE retail stores?". She'll use this bot to ask questions instead of manually scanning PDFs, saving 2 hours per day.

Step 1: Parse PDF with pypdf

pypdf extracts text from any PDF page-by-page. For a 10-Q, we'll preserve page numbers so we can cite them later.

parse_pdf.py Extract text from PDF, preserving page numbers
Total pages: 47

Page 1: 3254 characters
Page 2: 3891 characters
Page 3: 4120 characters
...

Page 1 preview:
FORM 10-Q
SOUTHBANK INDIA ADR
For the Quarter Ended June 30, 2024

PART I. FINANCIAL INFORMATION
Item 1. Consolidated Financial Statements...
Important:
We ship a sample 10-Q with the lesson. Download it from the file section below so you can follow along.

Step 2: Chunk text into overlapping windows

A 47-page 10-Q is too big to embed as one chunk. We'll split it into 500-token chunks with 50-token overlap, keeping page numbers in the metadata.

chunker.py Split text into chunks with overlap and metadata
Total chunks: 142

First chunk:
FORM 10-Q SOUTHBANK INDIA ADR For the Quarter Ended June 30,
2024 PART I. FINANCIAL INFORMATION Item 1. Consolidated Financial
Statements The following consolidated financial statements have been
prepared in accordance with U.S. GAAP and are presented on a
consolidated basis. Our Q1 revenue grew 12% year-over-year to โ‚น8,450
crore, driven by...

Step 3: Embed chunks and store in Chromadb

We'll use nomic-embed-text to generate embeddings for each chunk, then store them in Chromadb for fast vector search.

embed_and_store.py Generate embeddings with nomic and store in Chromadb
Stored 142 chunks in Chromadb
Note:
nomic-embed-text is 137M parameters and runs locally. For production, use paid APIs (OpenAI, Cohere) or self-host larger models. The embeddings are dimension 768.

Step 4: Retrieve relevant chunks on query

When the user asks a question, embed it and find the top-k most similar chunks from the database.

retrieve.py Query and retrieve top-k chunks with citations
Rank 1 (Page 3, Financial Results)
FORM 10-Q SOUTHBANK INDIA ADR Q1 2024 revenue was โ‚น8,450 crore,
representing a 12% increase year-over-year. Net interest income
grew 8% to โ‚น2,200 crore...

Rank 2 (Page 4, Consolidated Statements)
Total operating income for the quarter was โ‚น11,320 crore. Operating
expenses decreased 2% due to improved efficiency in back-office
operations. Cost-to-income ratio improved...

Rank 3 (Page 5, Management Discussion)
SouthBank's revenue growth was driven by strong performance in
the retail lending segment (โ‚น1,800 crore) and digital banking
(โ‚น950 crore)...

Step 5: Prompt LLM with context and extract citations

Feed the retrieved chunks to the LLM as context, asking it to answer and cite pages.

answer_with_citations.py LLM answers with page citations from context
Question: What is SouthBank's Q1 2024 revenue?

Answer: SouthBank reported Q1 2024 revenue of โ‚น8,450 crore,
representing a 12% increase year-over-year. [Page 3, 'Financial
Results']

Step 6: Multi-turn chat with context memory

Use ChatSession to remember previous turns, so follow-up questions can reference earlier answers.

filing_chat.py Full multi-turn filing Q&A bot
Loaded 142 chunks from southbank_10q.pdf
Ask questions about the filing (type 'quit' to exit)

You: What is SouthBank's Q1 revenue?
Bot: SouthBank reported Q1 2024 revenue of โ‚น8,450 crore.
[Page 3, 'Financial Results']

You: How does that compare to last year?
Bot: That represents a 12% year-over-year increase from โ‚น7,544
crore in Q1 2023. [Page 3, 'Financial Results']

You: Which segment drove the growth?
Bot: Retail lending grew strongly (โ‚น1,800 crore, +18%) and
digital banking contributed โ‚น950 crore (+25%). [Page 5,
'Segment Performance']

Handling documents where answers don't exist

The biggest RAG risk is hallucination โ€” the LLM making up an answer when the filing doesn't contain the info. Force the model to say "not found in filing".

Prompt technique:
Add to system prompt: "If the answer is not in the provided filing context, respond with exactly: 'This information is not available in the provided filing.'". Then check the response for that exact phrase. If present, tell the user the info wasn't found.

How RAG powers real AI systems

This filing bot is a real RAG system. In production, RAG powers customer service (answer questions from docs), legal research (search contracts), medical diagnosis (retrieve case studies), and enterprise search (find answers in Slack/Confluence). The key insight: LLMs are powerful reasoners but can't access external documents. RAG solves that by retrieving relevant chunks first, then reasoning over them. Every major AI company (OpenAI, Anthropic, Google) offers RAG APIs because it's the gap between LLM capability and real-world use.

Vector search Semantic matching Context retrieval Augmented generation Citation grounding
๐ŸŽฏ Mini Challenge: Ask 3 follow-ups

Task: Run the filing bot on the sample 10-Q. Ask these 3 questions in order:

1. What is the company's total revenue for Q1?

2. How many employees does it have?

3. Which was the largest revenue segment?

Verify that:

- The bot retrieves text citing page numbers

- Questions 2 and 3 reference context from earlier answers

- At least one question gets "not found in filing" (test it deliberately)

Hint: Run `python filing_chat.py southbank_10q.pdf` then type your questions. The bot will retrieve chunks and answer them. To test "not found", ask something unrelated to the 10-Q (e.g., "What are company's political donations?") โ€” if it's not in the filing, the bot should refuse to answer.

Test your knowledge

Question 1 of 5
What are the 3 main phases of the RAG pipeline shown in this lesson?
Question 2 of 5
Why use 500-token chunks with 50-token overlap instead of treating the whole PDF as one chunk?
Question 3 of 5
What is the purpose of nomic-embed-text in this pipeline?
Question 4 of 5
How does the bot prevent hallucination (making up facts not in the filing)?
Question 5 of 5
In a multi-turn conversation, why re-retrieve context for each new question instead of using the same chunks?
0
/5
Excellent! You've mastered RAG document Q&A.
Africa & India's Best AI Academy
๐Ÿค– AI-first ๐ŸŽ“ Placement Assistance

You've built a real RAG system.
Now deploy it at scale.

Join the Best AI Program: The AI-Era Agentic Engineer โ€” 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Agentic Engineer โ†’