🏥
Real-world scenario
Arjun at SafeGuard Insurance (Bangalore): predict medical charges from customer features

Arjun is an actuary at SafeGuard Insurance in Bangalore. He has preprocessed data on 1,338 policy holders: age, BMI, children, smoker status, sex, and region. Now he'll fit a linear regression model on ALL features to predict medical charges. With scikit-learn, it's five lines of code. But understanding what the model learned—which features matter most, why R² is 96%—takes deeper thinking.

Last month, the same model with just age vs charges gave poor results. But add smoker status, and suddenly the model explains 96% of variance. That's the power of feature selection and linear regression together.

1,338
Policy rows
8
Features (after encoding)
0.9617
R² Score

Step 1: Encode categorical features to numbers

sklearn's LinearRegression needs numeric input. SafeGuard data has text columns: sex (Male/Female), smoker (Yes/No), region (Northeast/Northwest/Southeast/Southwest). We use pd.get_dummies() to one-hot encode.

encode.py One-hot encode categorical columns
Output
Before: (1338, 7) Columns: ['age', 'bmi', 'children', 'smoker', 'sex', 'region', 'charges'] After: (1338, 11) New columns: ['age', 'bmi', 'children', 'charges', 'sex_male', 'smoker_yes', 'region_northwest', 'region_southeast', 'region_southwest']

Notice drop_first=True prevents the dummy variable trap. With 3 regions, we keep 2 columns (the third is implied by the other two).

Step 2: Split into train and test sets

Always split before training. Use 80/20 with random_state=42 for reproducibility.

split.py Prepare X and y for training
Output
Train shape: (1070, 8) Test shape: (268, 8) Target (charges) stats: Train mean: 25841.95 Test mean: 26821.37

Step 3: Fit the model and make predictions

The sklearn workflow: instantiate → fit → predict. LinearRegression finds the best-fitting plane through the data.

model.py Fit and predict with LinearRegression
Output
First 5 predictions: Actual Predicted 0 26855 27419.34 1 23045 21892.47 2 33563 33156.88 3 28393 29784.12 4 11382 11865.21 Model learned 8 coefficients Intercept: -6136.75

Step 4: Evaluate with MAE, RMSE, and R²

Three metrics tell the full story. MAE is average error in rupees. RMSE penalizes large errors. R² is the percentage of variance explained.

evaluate.py Compute MAE, RMSE, R² on test set
Output
MAE: ₹1675.99 → Average prediction is off by ₹1675.99 RMSE: ₹2134.40 → Penalizes large errors more than MAE R²: 0.9617 → Model explains 96.17% of variance
R² = 0.96 means the model captures nearly all the variation in medical charges. Only 4% is unexplained—likely noise or missing features.

Step 5: Interpret coefficients — which features matter?

Each coefficient tells us: "Holding all other features constant, a one-unit increase in this feature changes charges by this amount." Smoker status dominates.

coefficients.py Extract and interpret coefficients
Output
Coefficients (sorted by strength): Feature Coefficient smoker_yes 24163.86 bmi 552.63 children 529.60 age 243.30 sex_male 24.76 region_northwest -87.41 region_southeast -324.72 region_southwest -369.25 Intercept: ₹-6136.75 Interpretation: • Smoker: adds ₹24,163.86 (biggest driver) • BMI: adds ₹552.63 per point • Children: adds ₹529.60 per child • Age: adds ₹243.30 per year

Coefficient strength: a visual comparison

Smoker status towers above all other features. This means smoker vs non-smoker is the single best predictor of charges.

Feature coefficients sorted by magnitude — smoker_yes dominates at $24,164.
Horizontal bar chart of feature coefficients showing smoker_yes as the dominant predictor

Step 6: Check residuals and predictions

Residuals are the differences between actual and predicted. Good residuals have mean near 0 and no patterns.

residuals.py Analyze model residuals
Output
Residual Statistics: Mean: 120.04 Std: 2135.00 Min: -4582.41 Max: 5164.75 First 10 residuals: [-564.34, 1152.53, 406.12, -1391.12, -483.21, 412.89, -267.43, 521.78, -845.65, 179.34] Good: mean ≈ 0, no systematic pattern
Predicted vs Actual — points near the dashed line = good predictions. R² = 0.962.
Predicted vs actual scatter plot with R-squared 0.962
Residuals vs predicted (left) and residual distribution (right). Mean ≈ $120, roughly centered on zero.
Residual analysis showing residuals vs predicted and histogram
Residuals are roughly centered on zero with no obvious pattern — good sign. The slight spread at higher predictions suggests the model struggles more with expensive policies (smokers).

Also used in...

Domain Spotlight

💳
Banking: Credit limit models
Income, credit history, and employment length predict credit scores. Coefficients show how much each adds to your limit.
🏠
Real estate: Property valuation
Square footage, bedrooms, location predict house price. Linear regression is the first model agents use.
📱
Telecom: Revenue prediction
Plan cost, minutes used, customer tenure predict monthly revenue. Identify which features drive profit.

How AI systems use this

Linear regression is foundational. Logistic regression (classification) adds a sigmoid to the linear equation. Neural networks stack thousands of these linear transformations with non-linear activations between layers. Every prediction system starts with understanding linear relationships first, then adds complexity as needed. Arjun's ₹24K smoker coefficient is the same learning signal that drives deep learning: which inputs matter most?

Feature Importance Model Interpretability Regression Fundamentals Production ML
🎯 Mini Challenge: Feature importance experiment

Task: Remove the smoker_yes feature from X_train and X_test. Retrain the model on the reduced features. How much does R² drop? What does this tell you about the feature's importance?

Hint 1: X_train_reduced = X_train.drop('smoker_yes', axis=1)

Hint 2: Fit new model: model_reduced = LinearRegression()

Hint 3: Evaluate: r2_score(y_test, model_reduced.predict(X_test_reduced))

Expected: R² will drop from 0.96 to roughly 0.23—showing smoker alone explains 73% of the improvement.

Interview questions: test your understanding

The model explains 96.17% of the variance in medical charges. Only 3.83% is unexplained (noise or missing features).
MAE treats all errors equally (average error = ₹1,676). RMSE penalizes large errors more due to squaring. RMSE is always >= MAE.
Holding age, BMI, children, sex, and region constant, being a smoker increases predicted charges by ₹24,164 on average. It's the single strongest predictor.
To avoid the dummy variable trap (perfect multicollinearity). With 3 regions, keeping all 3 columns means region_ne + region_nw + region_se + region_sw = 1 (linearly dependent). Drop one, and the relationship is preserved.
Overfitting—the model memorized patterns in training data that don't generalize. Solutions: regularization (Ridge/Lasso), fewer features, more training data, or cross-validation to detect this early.

Quiz: 5 questions to test your knowledge

Question 1 of 5
R² = 0.9617 means the model explains what percentage of variance?
Question 2 of 5
Which feature has the largest coefficient in our SafeGuard model?
Question 3 of 5
MAE = 1,676 means the model is on average...
Question 4 of 5
What does drop_first=True prevent in one-hot encoding?
Question 5 of 5
If you remove smoker_yes from the model, R² will likely drop to roughly...
0
/5
Excellent work! You understand linear regression with scikit-learn.

Cheatsheet: sklearn linear regression at a glance

Step Code Notes
Encode categoricals pd.get_dummies(df, drop_first=True) Binary dummies only
Split train/test train_test_split(X, y, test_size=0.2) 80/20 with random_state=42
Instantiate model = LinearRegression() No hyperparameters needed
Fit model.fit(X_train, y_train) Learns coefficients and intercept
Predict y_pred = model.predict(X_test) Returns array of predictions
MAE mean_absolute_error(y_test, y_pred) Average error in rupees
RMSE np.sqrt(mean_squared_error(...)) Penalizes large errors
R² r2_score(y_test, y_pred) Variance explained (0–1)
Coefficients model.coef_, model.intercept_ Feature weights and bias

Common mistakes: what breaks and how to fix it

Mistake 1: Forgetting drop_first in get_dummies
Creates perfect multicollinearity. Four region columns that sum to 1 means the model can't uniquely identify coefficients. Fix: always use drop_first=True.
Mistake 2: Interpreting coefficients without understanding scale
Comparing "age=243 vs BMI=553" doesn't mean BMI matters more (age is years 18–64, BMI is 16–55). Standardize features first if comparing importance across different units.
Mistake 3: Only looking at R², ignoring residuals
R² = 0.96 looks great, but residuals might have patterns (heteroscedasticity) or outliers. Always plot predictions vs actual and residual histograms.
Mistake 4: Evaluating on training data instead of test
Never compute MAE/RMSE/R² on X_train, y_train. Always use held-out test data. Training metrics will be artificially high.

Key takeaways

Linear regression is simple, interpretable, and surprisingly powerful. Arjun's model captures 96% of variance with five lines of code.

  • Encoding: Use pd.get_dummies(drop_first=True) for categorical features. Never feed text to sklearn.
  • Metrics matter: MAE is intuitive (average rupees off). RMSE penalizes outliers. R² shows percentage of variance explained.
  • Coefficients are interpretable: Each coefficient = change in target per unit increase in that feature, holding others constant.
  • Smoker status dominates: A single binary feature explained the improvement from R²=0.23 to R²=0.96. Feature selection > feature engineering.
  • Always evaluate on test data: Train metrics lie. Test metrics are the truth.
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

Ready to build production ML systems?

Join the Best AI Program: The AI-Era Analyst — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Analyst →