🏬
Real-world scenario
TelcoMax (Nairobi): Mall customers cluster differently than circles

In Lesson 20, you used K-Means to segment TelcoMax mall customers into 5 round clusters. But K-Means assumes globular shapes. What if your customers form elongated chains? Hierarchical clustering builds a tree of merges (dendrogram), showing how points group at every level. What if you have outliers—customers who don't fit any normal cluster? DBSCAN finds them and flags them as noise, using density instead of distance.

Same 200 customers, new algorithms, new insights. Let's see how hierarchical and density-based methods reveal hidden patterns.

200
Mall customers
4
Linkage types
5
DBSCAN eps sweeps

Agglomerative Clustering: building trees bottom-up

K-Means starts with k centroids and iterates. Agglomerative does the opposite: start with 200 single-point clusters, then merge pairs until you have k clusters. The merging process is tracked in a dendrogram—a tree diagram showing which points/clusters combine and when.

Four linkage types define how to measure distance between clusters:
Ward: Minimize within-cluster variance (like K-Means). Most common.
Complete: Distance = farthest pair between clusters (compact clusters).
Average: Distance = mean of all pairwise distances (balanced).
Single: Distance = nearest pair (can create long chains).

Agglomerative Clustering with dendrograms

Let's apply agglomerative clustering to TelcoMax customers using Ward linkage and plot the dendrogram.

agglomerative.py Hierarchical clustering with dendrogram
Output
Cluster sizes: {0: 44, 1: 45, 2: 31, 3: 57, 4: 23} Silhouette: 0.4115

Agglomerative finds 5 natural clusters with silhouette 0.4115—similar to K-Means, but the dendrogram reveals the merging hierarchy, which K-Means doesn't.

Dendrogram: Ward linkage showing cluster merge history
Dendrogram with Ward linkage showing hierarchical merge structure

DBSCAN: density-based clustering for irregular shapes

DBSCAN doesn't assume round clusters. Instead, it groups points by density. A point is a core point if it has at least min_samples neighbors within radius eps. Neighbors of core points form clusters. Points in low-density regions become noise.

Three types of points:
Core point: At least min_samples points (including itself) within eps radius.
Border point: Not core, but within eps of a core point (assigned to its cluster).
Noise point: Neither core nor border (outlier, cluster label = -1).
Parameter Meaning How to choose
eps Radius of neighborhood search Too small: everything is noise. Too large: one big cluster. Plot k-distance graph; pick knee.
min_samples Points needed (including self) to be core Usually = dimensions + 1. For 2D: min_samples=3 or 5.

DBSCAN with eps sweep

DBSCAN is sensitive to eps. Let's sweep different eps values and observe cluster counts and noise points.

dbscan_sweep.py DBSCAN eps parameter sensitivity
Output
eps=0.3: clusters=5, noise=47, silhouette=0.0260 eps=0.5: clusters=1, noise=5, silhouette=nan eps=0.7: clusters=1, noise=1, silhouette=nan eps=1.0: clusters=1, noise=1, silhouette=nan

At eps=0.3: tight clustering, many outliers (47 noise points). At eps=0.5+: everything merges into one giant cluster. The sweet spot depends on your domain and what "outlier" means.

DBSCAN eps sweep: visualizing the trade-off

Clusters and noise points as eps increases (min_samples=5)
DBSCAN eps sweep showing cluster and noise point counts
At eps=0.3, DBSCAN detects 5 clusters with 47 outliers. As eps grows, points merge, clusters decrease, and outliers shrink. Choose eps based on how much outlier detection you need.

Hierarchical vs. DBSCAN vs. K-Means

Aspect K-Means Hierarchical (Agglomerative) DBSCAN
Cluster shape Round (convex) Any shape (via dendrogram) Any shape (density-based)
Number of clusters Fixed (k given) Choose by cutting dendrogram Emergent (from eps, min_samples)
Outliers Forced into clusters Assigned to nearest cluster Labeled as noise (-1)
Dendrogram No Yes (full merge history) No
Scalability Fast (O(nk) per iter) Slow (O(n²) or O(n² log n)) Slow (O(n²) without spatial index)

Also used in...

Domain Spotlight

🌐
Social Networks: Community detection
DBSCAN finds dense groups of connected users; hierarchical shows how communities merge. Identifies influencers (low-density nodes).
🗺️
GIS & Spatial Data: Geographic clusters
DBSCAN naturally clusters locations by proximity; irregular boundaries (neighborhoods, hotspots) emerge without forcing rounds.
🚨
Anomaly Detection: Fraud, intrusion
DBSCAN's noise points are outliers—flagged as potential fraud or attack. No need to pre-specify how many anomalies to expect.

How AI systems use this

Large language models don't cluster text directly, but clustering powers retrieval and recommendation. DBSCAN finds clusters of similar documents; hierarchical clustering helps organize knowledge hierarchically (like a taxonomy). Outlier detection via DBSCAN catches malformed inputs before they reach the model.

Hierarchical Models Anomaly Detection Data Organization Feature Similarity
🎯 Mini Challenge: DBSCAN on noisy data

Task: Load mall_customers.csv. Scale it. Apply DBSCAN with eps=0.4, min_samples=5. Count clusters (exclude noise). Print noise count. Compare: which eps (0.3, 0.4, 0.5) gives the best balance of cluster compactness and outlier detection?

Hint 1: Use DBSCAN(eps=0.4, min_samples=5).fit_predict(X_scaled)

Hint 2: Count noise: list(labels).count(-1)

Hint 3: Clusters: len(set(labels)) - (1 if -1 in labels else 0)

Hint 4: For 2D data, silhouette ignores noise: filter labels before computing.

Interview questions: test your understanding

Agglomerative: start with n clusters (each point), merge pairs until k. Divisive: start with 1 cluster, split until k. Agglomerative is more common and easier to compute.
Core: has ≥ min_samples neighbors within eps (including itself). Border: not core, but within eps of a core point. Noise: neither core nor border. Only core and border form clusters; noise gets label -1.
Plot the k-distance graph: sort distances to the kth nearest neighbor (k=min_samples-1), plot from farthest to nearest. Look for the "elbow" (knee). That distance is a good eps candidate. Or test multiple eps and inspect results.
A dendrogram shows the full merge history. You can "cut" it at different heights to get different numbers of clusters. You don't need to commit to k beforehand; you can explore the trade-offs visually.
23.5% of your data is sparse (low-density regions). Either your data has natural outliers (e.g., rare customer segments), or eps=0.3 is too tight. Increase eps to capture more points, or investigate whether those noise points are truly anomalous or just on cluster boundaries.

Quiz: 5 questions to test your knowledge

Question 1 of 5
In agglomerative clustering with Ward linkage, which metric is minimized at each merge?
Question 2 of 5
What is a core point in DBSCAN?
Question 3 of 5
In the eps sweep example, what happens as eps increases from 0.3 to 1.0?
Question 4 of 5
How do dendrograms in hierarchical clustering help you choose k (number of clusters)?
Question 5 of 5
When is DBSCAN better than K-Means for clustering?
0
/5
Excellent! You've mastered clustering hierarchies and densities.

Cheatsheet: hierarchical & density clustering at a glance

Algorithm Code Best for
Agglomerative (Ward) AgglomerativeClustering(n_clusters=k, linkage='ward') Exploring cluster hierarchies, moderate data
Dendrogram dendrogram(linkage(X, 'ward')) Visualizing merges, choosing k
DBSCAN DBSCAN(eps=0.5, min_samples=5).fit_predict(X) Irregular clusters, outlier detection
Linkage types linkage='ward'|'complete'|'average'|'single' Ward=variance, complete=compact, average=balanced, single=chains
Choose eps (DBSCAN) Plot k-distance graph, find elbow No universal rule; domain and data shape matter

Common mistakes: what breaks and how to fix it

Mistake 1: Using agglomerative on large datasets
Agglomerative is O(n²) or O(n² log n), slow for n > 10k. Fix: use K-Means or DBSCAN for big data, or subsample.
Mistake 2: Not scaling before DBSCAN
DBSCAN depends on distance; unscaled features with different ranges bias results. Fix: always StandardScaler before DBSCAN.
Mistake 3: Choosing eps arbitrarily
Guessing eps leads to all-noise or all-one-cluster results. Fix: plot k-distance graph or sweep eps and inspect silhouette/cluster counts.
Mistake 4: Ignoring noise points in evaluation
Computing silhouette on noise points gives misleading results. Fix: filter noise (labels != -1) before metric calculation.

Key takeaways

Clustering is not one-size-fits-all. Choose wisely based on your data and goals.

  • Hierarchical clustering: Build a tree of merges. Dendrograms let you explore cluster hierarchies without pre-specifying k.
  • Linkage types matter: Ward (variance), complete (compact), average (balanced), single (chains). Ward is most common.
  • DBSCAN finds irregular shapes: No assumption of round clusters. Naturally handles outliers (noise points).
  • eps is critical for DBSCAN: Too small = everything is noise. Too large = one giant cluster. Use k-distance graph or sweep.
  • Scale before density-based methods: DBSCAN depends on distance; unscaled features bias results.
  • K-Means vs. Hierarchical vs. DBSCAN: Use K-Means for speed and round clusters. Hierarchical for exploration. DBSCAN for irregular shapes and outliers.
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

Ready to master unsupervised learning?

Join the Best AI Program: The AI-Era Analyst — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Analyst →