NeoBank's risk team in Mumbai handles 10,000 loan applications. The question: will this applicant default? Unlike regression (predicting a loan amount), this is binary: Yes or No. Logistic regression outputs probability between 0 and 1.
Historical data shows: high credit scores reduce default risk. Large loans increase it. Low income increases it. We train a classifier, interpret coefficients, and optimize the threshold based on cost—because approving a defaulter costs ₹5 lakh, but rejecting a good customer costs revenue.
From regression to classification: the shift
Linear regression predicts continuous numbers. Logistic regression predicts probabilities for binary outcomes. Both use y = β₀ + β₁x₁ + β₂x₂ + ... But linear regression outputs this directly. Logistic regression wraps it in sigmoid, squashing to [0, 1].
The sigmoid function: probability squashing
The sigmoid is an S-curve. At z=0, σ(0)=0.5. Large positive z→σ(z)→1. Large negative z→σ(z)→0.
Decision boundary: from probability to class
Sigmoid outputs probability P(default). We need a class prediction. We set a threshold: if P ≥ 0.5, predict 1 (default); else 0 (safe). The threshold is tunable based on business costs.
Training logistic regression with sklearn
LogisticRegression learns via maximum likelihood estimation. It finds coefficients that maximize probability of the observed training data.
Confusion matrix and classification metrics
Accuracy is insufficient. The confusion matrix shows TN, FP, FN, TP. From it we compute Precision, Recall, F1, and AUC.
| Metric | Formula | Our Result | When to Use |
|---|---|---|---|
| Accuracy | (TP+TN)/Total | 0.7370 | Balanced classes |
| Precision | TP/(TP+FP) | 0.6868 | False alarm cost high |
| Recall | TP/(TP+FN) | 0.6444 | Missing positive costly |
| F1 | 2·P·R/(P+R) | 0.6650 | Balance P and R |
| AUC | Area under ROC | 0.8029 | Ranking ability |
Interpreting logistic coefficients
Each coefficient represents the change in log-odds for a 1 std-dev increase in that feature. Negative coefficients reduce default probability. Positive increase it.
Top coefficients: credit_score=-1.1123 (strongest), loan_amount=0.7328, income=-0.5461. A standard-dev increase in credit_score decreases log-odds of default by 1.11.
ROC curve and AUC: ranking ability
AUC=0.797 means: if you pick one random defaulter and one non-defaulter, there's ~80% chance the model ranks the defaulter higher. AUC=0.5 is random; AUC=1.0 is perfect.
Also used in...
Domain Spotlight
How AI systems use this
When Gmail marks spam, when your phone unlocks with FaceID, when Netflix shows a 97% match—they all use classification. Logistic regression is the simplest version. Deep neural networks are logistic regression stacked in layers. Transformer models use sigmoid at the output for ranking tasks. The sigmoid is everywhere.
Task: Train logistic regression on credit_default.csv. Predict probabilities on test set. Compute Precision and Recall at thresholds [0.3, 0.5, 0.7]. Which threshold gives highest Recall? Which gives highest Precision? What's the business trade-off?
y_prob = model.predict_proba(X_test)[:, 1]. Then manually compute: y_pred = (y_prob >= threshold).astype(int)Interview Q&A: test your understanding
Quiz: 5 questions
Cheatsheet: logistic regression at a glance
| Concept | Code | Notes |
|---|---|---|
| Train model | LogisticRegression().fit(X, y) | Always scale first |
| Predict proba | model.predict_proba(X)[:, 1] | Returns P(default) |
| Custom threshold | (y_prob >= 0.3).astype(int) | Tune for business cost |
| Metrics | precision_score, recall_score, roc_auc_score | sklearn.metrics |
| ROC curve | roc_curve(y, y_prob) | Threshold trade-off plot |
Common mistakes
Key takeaways
- Sigmoid maps to [0,1]: σ(z) = 1/(1+e^(-z)). Perfect for probabilities.
- Threshold is tunable: Default 0.5 isn't optimal. Choose based on business cost of FP vs FN.
- Metrics matter more than accuracy: Precision, Recall, F1, and AUC tell the real story with imbalanced data.
- AUC is robust: Ranks across all thresholds. Use it for model comparison.
- Always scale: LogisticRegression is linear. StandardScaler is essential.
- Coefficients are interpretable: credit_score=-1.11 means each std-dev increase reduces log-odds of default by 1.11.
Ready to deploy ML models?
Join the Best AI Program: The AI-Era Analyst — 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Analyst →