💰
Real-world scenario
SafeGuard Insurance: Build a production-ready premium predictor

Arjun is a data scientist at SafeGuard Insurance in Bangalore. The CEO wants a model that predicts customer premiums from health and lifestyle data. Not a prototype—a complete, documented pipeline the actuarial team can maintain. It must explain its predictions and be honest about error margins.

This lesson is different. You're not learning ONE technique. You're orchestrating several: EDA to find signals, preprocessing to clean, model selection, and rigorous evaluation. This is what production ML looks like.

1,338
Policies
7
Raw features
₹26K
Avg premium

The 4-phase ML pipeline

Production ML is not coding. It's asking the right questions at each step.

Phase 1: Understand — Load, visualize, correlate. What patterns exist?
Phase 2: Transform — Handle missing, encode categories, scale numerics.
Phase 3: Build — Try LinearRegression, Ridge, Lasso. Pick the best.
Phase 4: Evaluate — Test metrics, residual analysis, business interpretation.

Let's walk through each phase with SafeGuard's data.

Phase 1: Exploratory Data Analysis

Never jump to modeling. First understand your data. Load it, ask questions, find signals.

EDA visualizations: smoker impact on charges (boxplot) and BMI-charge relationship (scatter)
Boxplot comparing charges by smoker status and scatter plot of BMI vs charges
phase1_eda.py Load and inspect SafeGuard data
Output
Shape: (1338, 7) Data types: age int64 sex object bmi float64 children int64 smoker object region object charges float64 Missing: 0 Charges (premiums) stats: Mean: ₹26089.80 Median: ₹22654.75 Std: ₹11045.41 Smoker split: No: 1069 (79.9%) Yes: 269 (20.1%) Charges by smoker status: Yes: ₹45604 avg No: ₹21179 avg Correlations with charges: age: 0.305 bmi: 0.314 children: 0.026

Key insight: Smoker status creates a massive gap—₹24,425 difference on average. This single feature will dominate the model. Age and BMI correlate weakly with charges alone (0.305, 0.314), but combined with smoker status, they'll matter.

Phase 2: Preprocessing and feature engineering

The data is clean (no missing values), but it's mixed: numeric and categorical. We need a pipeline that encodes, scales, and produces model-ready arrays.

phase2_preprocess.py ColumnTransformer for mixed data types
Output
Original shape: (1338, 6) Processed shape: (1338, 9) First 5 processed rows: [[-0.63 0.14 -0.33 0. 1. 0. 0. 0. 1. ] [ 0.42 0.02 -0.33 0. 1. 0. 0. 1. 0. ] [ 0.32 -0.23 0.03 0. 1. 0. 1. 0. 0. ] [-0.52 -0.17 -0.33 0. 1. 0. 0. 0. 1. ] [ 1.08 0.08 -0.33 0. 1. 0. 0. 1. 0. ]]

ColumnTransformer handles both: StandardScaler on numerics (age, bmi, children), OneHotEncoder on categories (sex, smoker, region). Original 6 features become 9 after encoding. Numeric features are now centered at 0 with std=1.

Phase 3: Build, train, and compare models

Now we test three algorithms: LinearRegression (baseline), Ridge (light regularization), and Lasso (feature selection). Each gets the same preprocessed data.

Model comparison: R² scores across LinearRegression, Ridge, and Lasso algorithms
Bar chart comparing R-squared values for LinearRegression, Ridge, and Lasso models
phase3_models.py Compare LinearRegression, Ridge, Lasso