πŸ“ž
Real-world scenario
TelcoMax Nairobi: F1=0.009 baseline is terrible. Raw features aren't enough.

Samuel built a logistic regression model on TelcoMax's raw churn data: 5,000 customers, 13 columns. The baseline F1 score? 0.0087. Disaster. The model can barely detect churners. Raw features (tenure, monthly_charges, total_charges) lack domain insight. He needs to engineer new features: charge per month (domain knowledge), tenure Γ— charges (interactions), contract type β†’ churn rate (target encoding). With engineered features, F1 jumps to 0.0494 and AUC improves from 0.668 to 0.679.

5,000
Customers
11β†’18
Features (before→after)
0.668β†’0.679
AUC improvement

What is feature engineering?

Feature engineering is creating new predictive variables from raw data using domain knowledge. Raw features tell the model facts; engineered features tell it the story. A telecom model doesn't care that a customer paid $45 monthly and $2,200 total. It cares that they're paying $45/month, they're new (tenure 3 months), and customers on month-to-month contracts churn 33% of the time.

Why engineer features?
Raw data misses patterns. A model trained only on raw age, charges, and tenure can't spot that loyal, high-value customers (long tenure + high charges) rarely churn. But if you create a feature that captures this relationship, the model learns instantly.

Domain features: business logic into code

Domain features combine raw columns using business knowledge. In telecom, charge_per_month matters more than raw monthly_charges, because it normalizes by tenure. A customer paying $100/month for 60 months (loyal) is different from one paying $100/month for 3 months (risky).

domain_features.py Create business-driven features
Output
charge_per_month stats: Mean: 66.39 Std: 28.34 is_new_customer distribution: New (1): 421 (8.4%) Established (0): 4579 (91.6%)

Interaction terms: combining raw features

Interaction features capture how two variables work together. Tenure alone isn't predictive, charges alone isn't predictive. But tenure Γ— charges (total_charges) already exists! We can create more: tenure Γ— contract_type (do long contracts + long tenure = super loyal?), new_customer Γ— monthly_charges (are new customers price-sensitive?).

interactions.py Multiply and combine raw features
Output
Interaction features created: tenure_x_charges: mean=2536.7, std=2104.3 log_tenure: mean=3.11, std=1.42 tenure_to_age_ratio: mean=0.041, std=0.033 spend_per_age: mean=1.535, std=0.68

Binning: discretize continuous features

Sometimes a continuous variable works better as categories. Binning (also called bucketing) groups ranges into buckets: age 18–30 = young, 31–50 = mid, 51+ = senior. This captures non-linear relationships and reduces outlier impact.

binning.py Convert continuous to categorical buckets
Output
Tenure bucket distribution: new 421 mid 1889 loyal 2690 Name: tenure_bucket, dtype: int64 Age group distribution: young 1124 mid-age 2567 senior 1309 Name: age_group, dtype: int64

Target encoding: encode by churn rate

Target encoding replaces categorical values with their mean target (churn rate). Instead of contract = "Month-to-month" β†’ 0/1/2, replace it with the percentage of Month-to-month customers who churned: 0.329. This is powerful but risky: WARNING: fit on train data only to avoid leakage!

target_encoding.py Encode categorical by churn rate (fit on train only!)
Output
Target encoding (churn rate by contract type): Month-to-month: 0.3290 One year: 0.1086 Two year: 0.1463 Train shape after encoding: (4000, 12) Test shape after encoding: (4000, 12)
CRITICAL: Always fit target encoding on train data only. Encoding test data using test churn rates leaks information, inflating performance metrics. Use the mapping learned from train.

Feature importance: which features drive predictions?

After engineering features, ask: which ones matter? Random forests compute feature importance as the average decrease in impurity (gini) across all trees. This tells you which engineered features are actually useful.

importance.py Compute and plot feature importance from Random Forest
Output
Top 10 feature importances: feature importance monthly_charges 0.1225 charge_per_month 0.1215 total_charges 0.1194 tenure_x_charges 0.1179 age 0.1098 is_new_customer 0.0956 log_tenure 0.0834 tenure_months 0.0721 tenure_to_age_ratio 0.0523 gender 0.0178 Test accuracy: 0.7823
Top 10 engineered features by Random Forest importance
Bar chart of top 10 feature importances from Random Forest model

Before and after: the impact of feature engineering

Baseline model: logistic regression on raw 11 features. Engineered model: logistic regression + random forest on 18 features (raw + domain + interactions + target-encoded).

Metric Baseline LR (raw features) Engineered LR (18 features) Engineered RF (18 features) Improvement
F1 Score 0.0087 0.0494 0.1122 +0.0407 (LR)
AUC 0.6680 0.6793 0.7156 +0.0113 (LR)
Accuracy 0.8156 0.8223 0.7823 +0.0067 (LR)
Features 11 18 18 +7 features

Key insight: Engineered features boosted F1 by 468%. While accuracy only improved by 0.67%, F1 (which penalizes false negatives heavily) jumped dramatically. This is crucial in churn: missing a churner is far worse than false-alarming.

Also used in...

Domain Spotlight

πŸ’³
Finance: Credit risk scoring
Create debt-to-income ratio, payment-to-limit ratio, age of oldest account, days since last delinquency. Target-encode zip codes by default rate.
πŸ₯
Healthcare: Patient readmission prediction
BMI categories, comorbidity count (interaction of diagnoses), days-since-discharge. Target-encode hospital code by readmission rate.
πŸ›’
E-commerce: Customer lifetime value
Purchases per month, average order value, cart abandonment rate, days since last purchase. Bin RFM (recency, frequency, monetary) for clustering.

How AI systems use this

Feature engineering is how you teach ML models to see like a domain expert. Transformers and neural networks can learn representations automatically, but classical ML (trees, SVM, linear models) rely entirely on the features you feed them. The difference between 0.668 AUC and 0.679 AUC is pure domain knowledge converted to code. This human insight is irreplaceable.

Domain Knowledge Feature Creation Model Performance Business Impact
🎯 Mini Challenge: E-commerce feature engineering

Task: Load e-commerce data (product_price, user_purchases, user_age, category). Engineer: 1) average_purchase_value (price / purchases), 2) is_young (age < 30), 3) price_to_age ratio. Then target-encode category by conversion rate. Train RF and print top 5 feature importances.

Hint 1: Create average_purchase_value = product_price / (user_purchases + 1) to avoid division by zero.

Hint 2: One-hot or target-encode category after the split to avoid leakage.

Hint 3: RandomForestClassifier with n_estimators=100. Get importances from .feature_importances_

Hint 4: Print pd.DataFrame({'feature': X.columns, 'importance': rf.feature_importances_}).sort_values('importance', ascending=False).head(5)

Interview questions: test your understanding

Feature engineering is creating new predictive variables from raw data using domain knowledge. It matters because raw features often miss patterns. A churn model can't see that new customers on month-to-month contracts are at high risk until you engineer is_new_customer and contract_encoded features.
1) charge_per_month (total_charges / tenure) β€” normalizes spending by loyalty. 2) is_new_customer (tenure < 6 months) β€” new customers churn more. 3) is_high_value (monthly_charges >= 75th percentile) β€” high-value customers are treated better and churn less.
Target encoding leaks information: if you encode test data using test churn rates, the model learns from test data, inflating performance. Fix: fit the encoding on train data only, then apply the same mapping to test. Always split before encoding.
Start with domain knowledge: talk to telecom experts, ask what matters to churn (loyalty, contract type, spending patterns). Then look at raw feature correlations with target. Finally, test: engineer a feature, train the model, measure AUC. If it improves, keep it. If not, drop it.
Depends on context. In telecom, 0.011 AUC = ~50 more churners detected per 5,000 customers. If retention saves $200 per customer, that's $10k+ revenue. But model complexity increases maintenance. Decision: if business impact > engineering cost, engineer. If AUC improved but other metrics (precision, recall) got worse, investigate trade-offs.

Quiz: 5 questions to test your knowledge

Question 1 of 5
What is a domain feature in the context of telecom churn?
Question 2 of 5
Why is the interaction term tenure Γ— monthly_charges useful?
Question 3 of 5
What is the main risk of target encoding?
Question 4 of 5
In the TelcoMax example, month-to-month contracts were target-encoded to 0.3290. What does this number mean?
Question 5 of 5
Why did feature engineering boost TelcoMax's F1 score from 0.0087 to 0.0494 (468% improvement)?
0
/5
Excellent! You understand feature engineering.

Cheatsheet: feature engineering at a glance

Feature Type How to Create Example (Telecom)
Domain feature Combine raw columns using business logic charge_per_month = total_charges / (tenure + 1)
Interaction term Multiply or combine two features tenure_x_charges = tenure * monthly_charges
Binning Discretize continuous into buckets tenure_bucket = pd.cut(tenure, bins=[0,6,24,72])
Target encoding Replace categorical by target mean (fit on train only!) contract_encoded = contract.map(y_train.groupby(contract).mean())
Polynomial features Square, cube, or cross-multiply age_squared = age ** 2
Log transform Handle skewed distributions log_tenure = np.log(tenure + 1)
Ratio features Divide one feature by another spend_per_age = monthly_charges / age

Common mistakes: what breaks and how to fix it

Mistake 1: Engineering features on the full dataset before splitting
You leak test information into the model. Fix: split first, then engineer on train data only, and apply the same transformation to test.
Mistake 2: Target-encoding test data using test churn rates
The model sees test data statistics, inflating performance metrics. Fix: fit the encoding dictionary on train only, then map train and test using this dictionary.
Mistake 3: Engineering too many features without validation
Overfitting: 50 engineered features will fit noise. Fix: engineer features based on domain knowledge and business impact, then validate with cross-validation.
Mistake 4: Ignoring feature importance and keeping useless features
You increase model complexity without improving predictions. Fix: after training, check feature_importances_, drop low-importance features, and retrain.

Key takeaways

Feature engineering turns domain knowledge into predictions. It's the bridge between raw data and ML magic.

  • Domain features capture business logic: charge_per_month tells the model about spending relative to loyalty.
  • Interactions reveal relationships: tenure Γ— charges = total_charges, but explicitly shows the combined effect.
  • Binning discretizes continuous variables: Tenure buckets (new/mid/loyal) often outperform raw tenure in tree models.
  • Target encoding is powerful but risky: Fit on train, map both train and test. Never encode test data using test targets.
  • Feature importance tells you what matters: monthly_charges and charge_per_month dominated TelcoMax (0.1225 and 0.1215).
  • Start with domain experts: Ask telecom teams what drives churn, then code it up. Intuition + data beats pure statistics.
Africa & India's Best AI Academy
πŸ€– AI-first πŸŽ“ Placement Assistance

Master feature engineering and build real ML systems

Join the Best AI Program: The AI-Era Analyst β€” 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Analyst β†’