๐Ÿ“ What is File Handling?

File handling is how Python reads data from disk, processes it, and writes the results back. In real systems, data lives in files โ€” CSVs, logs, configs, databases exports. You need to open them, read line by line or all at once, transform the data, then close them properly.

with open(filename) as file:
data = file.read()

The with statement is Python's safest way โ€” it automatically closes the file even if something goes wrong. No dangling file handles. No data loss.

Two goals in file handling: read data reliably, and write cleaned results. CSV is the most common format in banking and fintech.

๐ŸŽฏ The 4 File Modes

r
open(file, 'r')
Read only. File must exist.
w
open(file, 'w')
Write (overwrites). Creates if missing.
a
open(file, 'a')
Append (add to end). No overwrite.
r+
open(file, 'r+')
Read and write. File must exist.

๐Ÿฆ The Business Scenario

๐Ÿฆ
Banking ยท SouthBank India
Daily Transaction Reconciliation

Every evening, SouthBank India in Mumbai receives yesterday's transaction log as a CSV file. The ops team needs to read it, flag any transactions over โ‚น50,000, check for encoding issues (some branches send UTF-8, others send Latin-1), and write a cleaned CSV with flagged rows. No file left open. No data corrupted. All 847 transactions processed and logged.

847
Daily transactions
4
File modes to learn
<5s
Processing time

๐Ÿ’ป Step 1: Open & Read with `with`

The with statement creates a context โ€” the file is guaranteed to close, whether your code crashes or not. No try/finally needed.

๐Ÿ“„
Sample data files to follow along
Save these next to your .py file so open('transactions.txt') works.
read_transactions.py โ–ถ Run to see output below
# Output Total lines: 847 File is safely closed ๐Ÿšฉ Flagged: TXN-2408,2024-01-15,75500.00,Mumbai ๐Ÿšฉ Flagged: TXN-2412,2024-01-15,62300.50,Delhi ...

๐Ÿ’ป Step 2: Write Cleaned Data to a New File

After reading and processing, write the results. Mode w overwrites (safe here, we want fresh output). Mode a appends to existing files.

write_flagged.py โ–ถ Creates new CSV
# Output โœ… Wrote 47 flagged transactions to flagged_transactions.txt

๐Ÿ’ป Step 3: Parse CSV with csv.DictReader (Safer)

Instead of splitting strings manually, use the csv module. DictReader parses headers automatically, returns each row as a dict. Handles commas inside quoted fields, newlines in cells, and encoding edge cases.

parse_with_dictreader.py โ–ถ Robust CSV parsing
# Output Found 47 high-value transactions TXN-2408: โ‚น75,500.00 from Mumbai TXN-2412: โ‚น62,300.50 from Delhi

๐Ÿ’ป Step 4: Write CSV with csv.DictWriter

Mirror of DictReader. Takes dicts, writes proper CSV rows. Handles quoting, newlines, and encoding so you don't have to.

write_with_dictwriter.py โ–ถ Writes clean CSV
# Output โœ… Flagged transactions written to flagged_transactions.csv

๐ŸŒ Also Used In

File Handling & CSV Across Every Domain

๐Ÿฆ
with open('daily_ledger.csv') as f: reconcile_accounts()
Banking โ€” every bank reads/writes CSVs for clearing, settlement, fraud checks
๐Ÿ“ฑ
import csv; reader = csv.DictReader(billing_export.csv)
Telecom โ€” billing systems load subscriber records and usage logs from CSV
๐Ÿšš
writer.writerow({'shipment_id': id, 'status': 'delivered'})
Logistics โ€” every package movement written to CSV, then uploaded to API
๐Ÿค–
pd.read_csv('training_data.csv', encoding='utf-8', sep=',')
AI & ML โ€” Pandas loads CSVs as training data, encoding matters for non-English text

๐Ÿค– AI & Machine Learning Connection

Every machine learning pipeline starts with CSV data. When you use Pandas df = pd.read_csv(..., encoding='utf-8'), it handles file I/O behind the scenes using exactly what you just learned. Large datasets use Parquet or HDF5 for speed, but 80% of ML starts with CSV. You'll also write model predictions back to CSV for reporting โ€” file handling never goes away.

pandas scikit-learn data preprocessing feature engineering

๐ŸŽฏ Mini Challenge

Process a Payout File from M-Pesa Kenya

Create or use an existing CSV file with agent payouts (amount in KSh, date, agent_name, status). Read it with csv.DictReader, find all payouts over 5,000 KSh that are still 'pending', and write them to a new CSV called urgent_payouts.csv with csv.DictWriter. Use with for both files and encoding='utf-8'.

Start with: with open('payouts.csv', 'r', ...) as infile, open('urgent_payouts.csv', 'w', ...) as outfile:. Create a DictReader from infile and DictWriter to outfile. Loop through rows, check if float(row['amount']) > 5000 and row['status'] == 'pending': then writer.writerow(row). Don't forget writeheader() at the start!

๐ŸŽค Interview Questions

The with statement is a context manager โ€” it guarantees the file closes, even if code crashes. Without with, you must call file.close() yourself. If an exception occurs before you close, the file handle stays open, corrupting data or exhausting the OS limit. In production, always use with. It's a Python best practice.
csv.reader returns each row as a list. You access columns by index: row[0], row[1]. csv.DictReader returns each row as a dict, using the header row as keys. You access columns by name: row['amount'], row['date']. DictReader is safer and more readable โ€” recommended for real data where column order might change.
On Windows, Python auto-converts \n (Unix newline) to \r\n (Windows CRLF) by default. The csv module expects raw bytes. Pass newline='' to disable auto-conversion. Otherwise, csv files get double-newlines on Windows. Always use newline='' when writing CSV. Reading also benefits from it to avoid blank rows.
Always specify encoding='utf-8' (default). If a file uses a different encoding, you'll get UnicodeDecodeError. Try encoding='latin-1' next. In production, use try/except to detect: try UTF-8, catch the error, fall back to Latin-1 or ask the user. Some systems use chardet library to auto-detect encoding. For CSV especially, ask the source what encoding they use โ€” saves debugging time.
It crashes. .readlines() loads the entire file into a list in RAM. A 10 GB file needs 10 GB of memory. Instead, iterate line by line: for line in file: or use csv.DictReader(file) which streams rows. For huge files, chunk them: read 100,000 rows at a time, process, discard, repeat. Pandas has read_csv(..., chunksize=10000) for exactly this. Production data pipelines always stream, never load-all.

๐Ÿง  Quick Quiz

Question 1 of 5
What mode do you use to read an existing file?
Question 2 of 5
What does readlines() return?
Question 3 of 5
Why do you write newline='' when opening CSV files for writing?
Question 4 of 5
Which code safely reads a CSV file with DictReader?
Question 5 of 5
What happens if you try to read a file that doesn't exist with mode 'r'?
0
/5

๐Ÿ“‹ Quick Cheatsheet

TaskCodeUse Case
Read all lineswith open(f) as f: lines = f.readlines()Small files only
Read line by linewith open(f) as f:
for line in f: ...
Large files, memory-safe
Read entire filewith open(f) as f: text = f.read()Text processing
Write file (new)with open(f, 'w') as f: f.write(...)Create or overwrite
Append to filewith open(f, 'a') as f: f.write(...)Add to existing file
Parse CSVcsv.DictReader(open(f, encoding='utf-8'))Read structured data
Write CSVcsv.DictWriter(f, fieldnames=[...], newline='')Save as CSV
Check file existsimport os; os.path.exists(f)Avoid FileNotFoundError

โš ๏ธ Common Mistakes

Forgetting to close the file
Always use with open(...) as f:. Never use bare open() without with. Open file handles leak and corrupt data. Production systems will hit OS limits and crash.
Using mode 'w' when you meant 'a'
Mode 'w' overwrites the file. Mode 'a' appends. If you need to add rows to an existing CSV, use 'a'. If you use 'w', all old data is gone instantly.
Forgetting newline='' with csv.DictWriter
On Windows, Python auto-converts newlines, creating double-newlines in CSV. Always write: open(f, 'w', newline='', encoding='utf-8'). The csv module expects raw bytes.
Loading huge files with .readlines()
A 5 GB file needs 5 GB RAM. Use iteration: for line in file: or csv.DictReader which streams. For Pandas, use chunksize=10000. Stream, don't load-all.

โœฆ Key Takeaways

  • Always use with open(...) as f: โ€” it auto-closes, even if code crashes
  • Know the 4 modes: r (read), w (write/overwrite), a (append), r+ (read-write)
  • For large files, iterate line-by-line โ€” don't use .readlines() on 10 GB files
  • Use csv.DictReader and csv.DictWriter, not raw string splitting โ€” they handle quotes, newlines, and encoding
  • When writing CSV, always use newline='' to prevent Windows double-newlines
  • Specify encoding='utf-8' explicitly โ€” real data comes in mixed encodings
  • Stream large datasets, never load all into memory โ€” the difference between working code and crashed servers
Africa & India's Best AI Academy
๐Ÿค– AI-first ๐ŸŽ“ Placement Assistance

You've processed CSV files.
Now go from reconciliation to revenue.

Join the Best AI Program: The AI-Era Analyst โ€” 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Analyst โ†’