The goal: conversational filing chat
Over the last three modules, you learned embeddings, vector databases, and multi-turn chat. Now you'll tie them together into a real-world document Q&A system.
In this lesson, you'll build a CLI bot that works like this:
$ python filing_chat.py southbank_10q.pdf Loading filing: southbank_10q.pdf Embedding and indexing... done. You: What is SouthBank's revenue for Q1 2024? Bot: SouthBank reported Q1 2024 revenue of โน8,450 crore, a 12% year-over-year increase. [Page 3, 'Financial Results'] You: How many branches do they operate? Bot: SouthBank operates 2,847 branches across India. [Page 7, 'Branch Network'] You: Compare that to the previous year. Bot: The filing shows 2,731 branches last year, so they added 116 branches in Q1. [Page 6, 'Year-over-year comparison']
The bot remembers every question you ask, retrieves relevant chunks from the PDF, and cites the page number so you can verify.
The 7-step RAG pipeline: parse โ chunk โ embed โ store โ retrieve โ prompt โ answer
Every RAG system follows the same flow: turn a document into chunks, embed them, store them in a vector database, retrieve the closest matches to a query, and feed them to an LLM.
Priya is an analyst at a Mumbai investment firm. Every day, she reads 5-10 company 10-Qs and answers stakeholder questions: "What is HDFC's Q2 NIM?" "How many RELIANCE retail stores?". She'll use this bot to ask questions instead of manually scanning PDFs, saving 2 hours per day.
Step 1: Parse PDF with pypdf
pypdf extracts text from any PDF page-by-page. For a 10-Q, we'll preserve page numbers so we can cite them later.
Total pages: 47 Page 1: 3254 characters Page 2: 3891 characters Page 3: 4120 characters ... Page 1 preview: FORM 10-Q SOUTHBANK INDIA ADR For the Quarter Ended June 30, 2024 PART I. FINANCIAL INFORMATION Item 1. Consolidated Financial Statements...
Step 2: Chunk text into overlapping windows
A 47-page 10-Q is too big to embed as one chunk. We'll split it into 500-token chunks with 50-token overlap, keeping page numbers in the metadata.
Total chunks: 142 First chunk: FORM 10-Q SOUTHBANK INDIA ADR For the Quarter Ended June 30, 2024 PART I. FINANCIAL INFORMATION Item 1. Consolidated Financial Statements The following consolidated financial statements have been prepared in accordance with U.S. GAAP and are presented on a consolidated basis. Our Q1 revenue grew 12% year-over-year to โน8,450 crore, driven by...
Step 3: Embed chunks and store in Chromadb
We'll use nomic-embed-text to generate embeddings for each chunk, then store them in Chromadb for fast vector search.
Stored 142 chunks in Chromadb
Step 4: Retrieve relevant chunks on query
When the user asks a question, embed it and find the top-k most similar chunks from the database.
Rank 1 (Page 3, Financial Results) FORM 10-Q SOUTHBANK INDIA ADR Q1 2024 revenue was โน8,450 crore, representing a 12% increase year-over-year. Net interest income grew 8% to โน2,200 crore... Rank 2 (Page 4, Consolidated Statements) Total operating income for the quarter was โน11,320 crore. Operating expenses decreased 2% due to improved efficiency in back-office operations. Cost-to-income ratio improved... Rank 3 (Page 5, Management Discussion) SouthBank's revenue growth was driven by strong performance in the retail lending segment (โน1,800 crore) and digital banking (โน950 crore)...
Step 5: Prompt LLM with context and extract citations
Feed the retrieved chunks to the LLM as context, asking it to answer and cite pages.
Question: What is SouthBank's Q1 2024 revenue? Answer: SouthBank reported Q1 2024 revenue of โน8,450 crore, representing a 12% increase year-over-year. [Page 3, 'Financial Results']
Step 6: Multi-turn chat with context memory
Use ChatSession to remember previous turns, so follow-up questions can reference earlier answers.
Loaded 142 chunks from southbank_10q.pdf Ask questions about the filing (type 'quit' to exit) You: What is SouthBank's Q1 revenue? Bot: SouthBank reported Q1 2024 revenue of โน8,450 crore. [Page 3, 'Financial Results'] You: How does that compare to last year? Bot: That represents a 12% year-over-year increase from โน7,544 crore in Q1 2023. [Page 3, 'Financial Results'] You: Which segment drove the growth? Bot: Retail lending grew strongly (โน1,800 crore, +18%) and digital banking contributed โน950 crore (+25%). [Page 5, 'Segment Performance']
Handling documents where answers don't exist
The biggest RAG risk is hallucination โ the LLM making up an answer when the filing doesn't contain the info. Force the model to say "not found in filing".
How RAG powers real AI systems
This filing bot is a real RAG system. In production, RAG powers customer service (answer questions from docs), legal research (search contracts), medical diagnosis (retrieve case studies), and enterprise search (find answers in Slack/Confluence). The key insight: LLMs are powerful reasoners but can't access external documents. RAG solves that by retrieving relevant chunks first, then reasoning over them. Every major AI company (OpenAI, Anthropic, Google) offers RAG APIs because it's the gap between LLM capability and real-world use.
Task: Run the filing bot on the sample 10-Q. Ask these 3 questions in order:
1. What is the company's total revenue for Q1?
2. How many employees does it have?
3. Which was the largest revenue segment?
Verify that:
- The bot retrieves text citing page numbers
- Questions 2 and 3 reference context from earlier answers
- At least one question gets "not found in filing" (test it deliberately)
Hint: Run `python filing_chat.py southbank_10q.pdf` then type your questions. The bot will retrieve chunks and answer them. To test "not found", ask something unrelated to the 10-Q (e.g., "What are company's political donations?") โ if it's not in the filing, the bot should refuse to answer.
Test your knowledge
You've built a real RAG system.
Now deploy it at scale.
Join the Best AI Program: The AI-Era Agentic Engineer โ 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Agentic Engineer โ