๐ What is File Handling?
File handling is how Python reads data from disk, processes it, and writes the results back. In real systems, data lives in files โ CSVs, logs, configs, databases exports. You need to open them, read line by line or all at once, transform the data, then close them properly.
data = file.read()
The with statement is Python's safest way โ it automatically closes the file even if something goes wrong. No dangling file handles. No data loss.
Two goals in file handling: read data reliably, and write cleaned results. CSV is the most common format in banking and fintech.
๐ฏ The 4 File Modes
๐ฆ The Business Scenario
Every evening, SouthBank India in Mumbai receives yesterday's transaction log as a CSV file. The ops team needs to read it, flag any transactions over โน50,000, check for encoding issues (some branches send UTF-8, others send Latin-1), and write a cleaned CSV with flagged rows. No file left open. No data corrupted. All 847 transactions processed and logged.
๐ป Step 1: Open & Read with `with`
The with statement creates a context โ the file is guaranteed to close, whether your code crashes or not. No try/finally needed.
open('transactions.txt') works.๐ป Step 2: Write Cleaned Data to a New File
After reading and processing, write the results. Mode w overwrites (safe here, we want fresh output). Mode a appends to existing files.
๐ป Step 3: Parse CSV with csv.DictReader (Safer)
Instead of splitting strings manually, use the csv module. DictReader parses headers automatically, returns each row as a dict. Handles commas inside quoted fields, newlines in cells, and encoding edge cases.
๐ป Step 4: Write CSV with csv.DictWriter
Mirror of DictReader. Takes dicts, writes proper CSV rows. Handles quoting, newlines, and encoding so you don't have to.
๐ Also Used In
File Handling & CSV Across Every Domain
๐ค AI & Machine Learning Connection
Every machine learning pipeline starts with CSV data. When you use Pandas df = pd.read_csv(..., encoding='utf-8'), it handles file I/O behind the scenes using exactly what you just learned. Large datasets use Parquet or HDF5 for speed, but 80% of ML starts with CSV. You'll also write model predictions back to CSV for reporting โ file handling never goes away.
๐ฏ Mini Challenge
Create or use an existing CSV file with agent payouts (amount in KSh, date, agent_name, status). Read it with csv.DictReader, find all payouts over 5,000 KSh that are still 'pending', and write them to a new CSV called urgent_payouts.csv with csv.DictWriter. Use with for both files and encoding='utf-8'.
with open('payouts.csv', 'r', ...) as infile, open('urgent_payouts.csv', 'w', ...) as outfile:. Create a DictReader from infile and DictWriter to outfile. Loop through rows, check if float(row['amount']) > 5000 and row['status'] == 'pending': then writer.writerow(row). Don't forget writeheader() at the start!
๐ค Interview Questions
with statement is a context manager โ it guarantees the file closes, even if code crashes. Without with, you must call file.close() yourself. If an exception occurs before you close, the file handle stays open, corrupting data or exhausting the OS limit. In production, always use with. It's a Python best practice.csv.reader returns each row as a list. You access columns by index: row[0], row[1]. csv.DictReader returns each row as a dict, using the header row as keys. You access columns by name: row['amount'], row['date']. DictReader is safer and more readable โ recommended for real data where column order might change.\n (Unix newline) to \r\n (Windows CRLF) by default. The csv module expects raw bytes. Pass newline='' to disable auto-conversion. Otherwise, csv files get double-newlines on Windows. Always use newline='' when writing CSV. Reading also benefits from it to avoid blank rows.encoding='utf-8' (default). If a file uses a different encoding, you'll get UnicodeDecodeError. Try encoding='latin-1' next. In production, use try/except to detect: try UTF-8, catch the error, fall back to Latin-1 or ask the user. Some systems use chardet library to auto-detect encoding. For CSV especially, ask the source what encoding they use โ saves debugging time..readlines() loads the entire file into a list in RAM. A 10 GB file needs 10 GB of memory. Instead, iterate line by line: for line in file: or use csv.DictReader(file) which streams rows. For huge files, chunk them: read 100,000 rows at a time, process, discard, repeat. Pandas has read_csv(..., chunksize=10000) for exactly this. Production data pipelines always stream, never load-all.๐ง Quick Quiz
readlines() return?newline='' when opening CSV files for writing?๐ Quick Cheatsheet
| Task | Code | Use Case |
|---|---|---|
| Read all lines | with open(f) as f: lines = f.readlines() | Small files only |
| Read line by line | with open(f) as f: | Large files, memory-safe |
| Read entire file | with open(f) as f: text = f.read() | Text processing |
| Write file (new) | with open(f, 'w') as f: f.write(...) | Create or overwrite |
| Append to file | with open(f, 'a') as f: f.write(...) | Add to existing file |
| Parse CSV | csv.DictReader(open(f, encoding='utf-8')) | Read structured data |
| Write CSV | csv.DictWriter(f, fieldnames=[...], newline='') | Save as CSV |
| Check file exists | import os; os.path.exists(f) | Avoid FileNotFoundError |
โ ๏ธ Common Mistakes
with open(...) as f:. Never use bare open() without with. Open file handles leak and corrupt data. Production systems will hit OS limits and crash.newline='' with csv.DictWriteropen(f, 'w', newline='', encoding='utf-8'). The csv module expects raw bytes.for line in file: or csv.DictReader which streams. For Pandas, use chunksize=10000. Stream, don't load-all.โฆ Key Takeaways
- Always use
with open(...) as f:โ it auto-closes, even if code crashes - Know the 4 modes:
r(read),w(write/overwrite),a(append),r+(read-write) - For large files, iterate line-by-line โ don't use
.readlines()on 10 GB files - Use
csv.DictReaderandcsv.DictWriter, not raw string splitting โ they handle quotes, newlines, and encoding - When writing CSV, always use
newline=''to prevent Windows double-newlines - Specify
encoding='utf-8'explicitly โ real data comes in mixed encodings - Stream large datasets, never load all into memory โ the difference between working code and crashed servers
You've processed CSV files.
Now go from reconciliation to revenue.
Join the Best AI Program: The AI-Era Analyst โ 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Analyst โ