Samuel built a logistic regression model on TelcoMax's raw churn data: 5,000 customers, 13 columns. The baseline F1 score? 0.0087. Disaster. The model can barely detect churners. Raw features (tenure, monthly_charges, total_charges) lack domain insight. He needs to engineer new features: charge per month (domain knowledge), tenure Γ charges (interactions), contract type β churn rate (target encoding). With engineered features, F1 jumps to 0.0494 and AUC improves from 0.668 to 0.679.
What is feature engineering?
Feature engineering is creating new predictive variables from raw data using domain knowledge. Raw features tell the model facts; engineered features tell it the story. A telecom model doesn't care that a customer paid $45 monthly and $2,200 total. It cares that they're paying $45/month, they're new (tenure 3 months), and customers on month-to-month contracts churn 33% of the time.
Raw data misses patterns. A model trained only on raw age, charges, and tenure can't spot that loyal, high-value customers (long tenure + high charges) rarely churn. But if you create a feature that captures this relationship, the model learns instantly.
Domain features: business logic into code
Domain features combine raw columns using business knowledge. In telecom, charge_per_month matters more than raw monthly_charges, because it normalizes by tenure. A customer paying $100/month for 60 months (loyal) is different from one paying $100/month for 3 months (risky).
Interaction terms: combining raw features
Interaction features capture how two variables work together. Tenure alone isn't predictive, charges alone isn't predictive. But tenure Γ charges (total_charges) already exists! We can create more: tenure Γ contract_type (do long contracts + long tenure = super loyal?), new_customer Γ monthly_charges (are new customers price-sensitive?).
Binning: discretize continuous features
Sometimes a continuous variable works better as categories. Binning (also called bucketing) groups ranges into buckets: age 18β30 = young, 31β50 = mid, 51+ = senior. This captures non-linear relationships and reduces outlier impact.
Target encoding: encode by churn rate
Target encoding replaces categorical values with their mean target (churn rate). Instead of contract = "Month-to-month" β 0/1/2, replace it with the percentage of Month-to-month customers who churned: 0.329. This is powerful but risky: WARNING: fit on train data only to avoid leakage!
Feature importance: which features drive predictions?
After engineering features, ask: which ones matter? Random forests compute feature importance as the average decrease in impurity (gini) across all trees. This tells you which engineered features are actually useful.
Before and after: the impact of feature engineering
Baseline model: logistic regression on raw 11 features. Engineered model: logistic regression + random forest on 18 features (raw + domain + interactions + target-encoded).
| Metric | Baseline LR (raw features) | Engineered LR (18 features) | Engineered RF (18 features) | Improvement |
|---|---|---|---|---|
| F1 Score | 0.0087 | 0.0494 | 0.1122 | +0.0407 (LR) |
| AUC | 0.6680 | 0.6793 | 0.7156 | +0.0113 (LR) |
| Accuracy | 0.8156 | 0.8223 | 0.7823 | +0.0067 (LR) |
| Features | 11 | 18 | 18 | +7 features |
Key insight: Engineered features boosted F1 by 468%. While accuracy only improved by 0.67%, F1 (which penalizes false negatives heavily) jumped dramatically. This is crucial in churn: missing a churner is far worse than false-alarming.
Also used in...
Domain Spotlight
How AI systems use this
Feature engineering is how you teach ML models to see like a domain expert. Transformers and neural networks can learn representations automatically, but classical ML (trees, SVM, linear models) rely entirely on the features you feed them. The difference between 0.668 AUC and 0.679 AUC is pure domain knowledge converted to code. This human insight is irreplaceable.
Task: Load e-commerce data (product_price, user_purchases, user_age, category). Engineer: 1) average_purchase_value (price / purchases), 2) is_young (age < 30), 3) price_to_age ratio. Then target-encode category by conversion rate. Train RF and print top 5 feature importances.
Hint 1: Create average_purchase_value = product_price / (user_purchases + 1) to avoid division by zero.
Hint 2: One-hot or target-encode category after the split to avoid leakage.
Hint 3: RandomForestClassifier with n_estimators=100. Get importances from .feature_importances_
Hint 4: Print pd.DataFrame({'feature': X.columns, 'importance': rf.feature_importances_}).sort_values('importance', ascending=False).head(5)
Interview questions: test your understanding
Quiz: 5 questions to test your knowledge
Cheatsheet: feature engineering at a glance
| Feature Type | How to Create | Example (Telecom) |
|---|---|---|
| Domain feature | Combine raw columns using business logic | charge_per_month = total_charges / (tenure + 1) |
| Interaction term | Multiply or combine two features | tenure_x_charges = tenure * monthly_charges |
| Binning | Discretize continuous into buckets | tenure_bucket = pd.cut(tenure, bins=[0,6,24,72]) |
| Target encoding | Replace categorical by target mean (fit on train only!) | contract_encoded = contract.map(y_train.groupby(contract).mean()) |
| Polynomial features | Square, cube, or cross-multiply | age_squared = age ** 2 |
| Log transform | Handle skewed distributions | log_tenure = np.log(tenure + 1) |
| Ratio features | Divide one feature by another | spend_per_age = monthly_charges / age |
Common mistakes: what breaks and how to fix it
Key takeaways
Feature engineering turns domain knowledge into predictions. It's the bridge between raw data and ML magic.
- Domain features capture business logic: charge_per_month tells the model about spending relative to loyalty.
- Interactions reveal relationships: tenure Γ charges = total_charges, but explicitly shows the combined effect.
- Binning discretizes continuous variables: Tenure buckets (new/mid/loyal) often outperform raw tenure in tree models.
- Target encoding is powerful but risky: Fit on train, map both train and test. Never encode test data using test targets.
- Feature importance tells you what matters: monthly_charges and charge_per_month dominated TelcoMax (0.1225 and 0.1215).
- Start with domain experts: Ask telecom teams what drives churn, then code it up. Intuition + data beats pure statistics.
Master feature engineering and build real ML systems
Join the Best AI Program: The AI-Era Analyst β 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Analyst β