🏦
Real-world scenario
NeoBank (Mumbai): 10,000 applicants, predict defaults

NeoBank's risk team in Mumbai handles 10,000 loan applications. The question: will this applicant default? Unlike regression (predicting a loan amount), this is binary: Yes or No. Logistic regression outputs probability between 0 and 1.

Historical data shows: high credit scores reduce default risk. Large loans increase it. Low income increases it. We train a classifier, interpret coefficients, and optimize the threshold based on cost—because approving a defaulter costs ₹5 lakh, but rejecting a good customer costs revenue.

10K
Applicants
40.5%
Default rate
0.8029
Model AUC

From regression to classification: the shift

Linear regression predicts continuous numbers. Logistic regression predicts probabilities for binary outcomes. Both use y = β₀ + β₁x₁ + β₂x₂ + ... But linear regression outputs this directly. Logistic regression wraps it in sigmoid, squashing to [0, 1].

Why sigmoid? We need outputs between 0 and 1 for probabilities. The sigmoid function σ(z) = 1/(1+e^(-z)) maps (-∞, ∞) to (0, 1).

The sigmoid function: probability squashing

The sigmoid is an S-curve. At z=0, σ(0)=0.5. Large positive z→σ(z)→1. Large negative z→σ(z)→0.

sigmoid.pyProperties of sigmoid curve
Output
σ(-2) = 0.1192 σ(0) = 0.5000 σ(2) = 0.8808
The sigmoid S-curve squashes any input into (0, 1). Above 0.5 → predict default; below → predict safe.
Sigmoid function curve with threshold at 0.5

Decision boundary: from probability to class

Sigmoid outputs probability P(default). We need a class prediction. We set a threshold: if P ≥ 0.5, predict 1 (default); else 0 (safe). The threshold is tunable based on business costs.

Threshold choice: High threshold (0.7): conservative, fewer false alarms. Low threshold (0.3): aggressive, more defaults caught. Choose based on cost of missing a default vs rejecting a good customer.

Training logistic regression with sklearn

LogisticRegression learns via maximum likelihood estimation. It finds coefficients that maximize probability of the observed training data.

train_model.pyTrain and evaluate
Output
Accuracy: 0.7370

Confusion matrix and classification metrics

Accuracy is insufficient. The confusion matrix shows TN, FP, FN, TP. From it we compute Precision, Recall, F1, and AUC.

MetricFormulaOur ResultWhen to Use
Accuracy(TP+TN)/Total0.7370Balanced classes
PrecisionTP/(TP+FP)0.6868False alarm cost high
RecallTP/(TP+FN)0.6444Missing positive costly
F12·P·R/(P+R)0.6650Balance P and R
AUCArea under ROC0.8029Ranking ability
Confusion matrix: 951 TN, 507 TP, 303 FN (missed defaults), 239 FP (false alarms).
Confusion matrix heatmap for logistic regression

Interpreting logistic coefficients

Each coefficient represents the change in log-odds for a 1 std-dev increase in that feature. Negative coefficients reduce default probability. Positive increase it.

Top coefficients: credit_score=-1.1123 (strongest), loan_amount=0.7328, income=-0.5461. A standard-dev increase in credit_score decreases log-odds of default by 1.11.

Coefficient direction: green features reduce default risk, red increase it.
Logistic regression coefficient bar chart

ROC curve and AUC: ranking ability

AUC=0.797 means: if you pick one random defaulter and one non-defaulter, there's ~80% chance the model ranks the defaulter higher. AUC=0.5 is random; AUC=1.0 is perfect.

ROC curve: the further from the diagonal, the better the model discriminates. AUC = 0.797.
ROC curve with AUC 0.797
AUC = 0.797 is decent for a baseline. Tree-based models (next lessons) will push this higher.

Also used in...

Domain Spotlight

💳
Banking: Fraud detection models
Classify transaction as fraud or legitimate. Threshold tuned for business cost (false alarm vs missed fraud).
🏥
Healthcare: Disease diagnosis
Binary outcome: disease present or absent. Optimize threshold based on cost of false negative (missed diagnosis).
📧
Email: Spam classification
Gmail uses logistic regression and ensembles to predict P(spam). Threshold tuned to avoid false positives (blocking legitimate mail).

How AI systems use this

When Gmail marks spam, when your phone unlocks with FaceID, when Netflix shows a 97% match—they all use classification. Logistic regression is the simplest version. Deep neural networks are logistic regression stacked in layers. Transformer models use sigmoid at the output for ranking tasks. The sigmoid is everywhere.

Binary Classification Probability Calibration Threshold Optimization ROC Analysis
Mini Challenge: Adjust threshold and measure trade-off

Task: Train logistic regression on credit_default.csv. Predict probabilities on test set. Compute Precision and Recall at thresholds [0.3, 0.5, 0.7]. Which threshold gives highest Recall? Which gives highest Precision? What's the business trade-off?

Use y_prob = model.predict_proba(X_test)[:, 1]. Then manually compute: y_pred = (y_prob >= threshold).astype(int)

Interview Q&A: test your understanding

σ(z) = 1/(1+e^(-z)). It maps any number to [0, 1], making it perfect for probabilities. At z=0, σ(0)=0.5.
Probability that the model ranks a random defaulter higher than a non-defaulter. AUC=0.8 means 80% chance. AUC=0.5 is random guessing.
With 40% default rate, a model predicting "no default" always gets 60% accuracy but catches zero defaults. Precision, Recall, and F1 are better.
Based on business cost. If missing a default (₹5L loss) costs more than rejecting a good customer, use low threshold (0.3). If vice versa, use high threshold (0.7).
Logistic regression is a linear model. Without scaling, large-magnitude features dominate the solution. StandardScaler ensures all features contribute fairly to the decision boundary.

Quiz: 5 questions

Question 1 of 5
What does sigmoid map?
Question 2 of 5
AUC=0.80 means?
Question 3 of 5
Precision vs Recall: which matters more when missing a default is costly?
Question 4 of 5
Why scale features before logistic regression?
Question 5 of 5
Our model: Accuracy=0.737, Precision=0.687, Recall=0.644. Why the gaps?
0
/5
Great job! You understand logistic regression.

Cheatsheet: logistic regression at a glance

ConceptCodeNotes
Train modelLogisticRegression().fit(X, y)Always scale first
Predict probamodel.predict_proba(X)[:, 1]Returns P(default)
Custom threshold(y_prob >= 0.3).astype(int)Tune for business cost
Metricsprecision_score, recall_score, roc_auc_scoresklearn.metrics
ROC curveroc_curve(y, y_prob)Threshold trade-off plot

Common mistakes

Mistake 1: Not scaling features
Logistic regression is linear. Unscaled features with different magnitudes bias the model. Always use StandardScaler.
Mistake 2: Using accuracy on imbalanced data
60% accuracy on 60/40 split might be predicting majority class always. Use Precision, Recall, F1, and AUC.
Mistake 3: Default threshold of 0.5
Business costs are asymmetric. Missing a default (₹5L) vs rejecting a customer (lost revenue) may favor different thresholds.
Mistake 4: Interpreting coefficients without scaling
Raw coefficients mix feature scale with importance. Fit on scaled data to get interpretable coefficients.

Key takeaways

  • Sigmoid maps to [0,1]: σ(z) = 1/(1+e^(-z)). Perfect for probabilities.
  • Threshold is tunable: Default 0.5 isn't optimal. Choose based on business cost of FP vs FN.
  • Metrics matter more than accuracy: Precision, Recall, F1, and AUC tell the real story with imbalanced data.
  • AUC is robust: Ranks across all thresholds. Use it for model comparison.
  • Always scale: LogisticRegression is linear. StandardScaler is essential.
  • Coefficients are interpretable: credit_score=-1.11 means each std-dev increase reduces log-odds of default by 1.11.
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

Ready to deploy ML models?

Join the Best AI Program: The AI-Era Analyst — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Analyst →