Clustering: K-Means and Hierarchical With easy examples.

Clustering groups data points so that points in the same group are more similar to each other than to points in other groups. Unlike classification, there are no pre-given labels — the algorithm discovers the groups on its own, making clustering a core technique of unsupervised learning. Two classic approaches are k-means and hierarchical clustering.

It matters because raw data rarely arrives organised. Businesses cluster customers into market segments, biologists cluster genes into functional families, and search engines cluster documents by topic — turning an undifferentiated pile of data into structure humans can reason about.

Key Concepts

• Cluster: a group of similar points. Good clustering gives tight, well-separated clusters.
• Centroid: the mean position of a cluster's points; in k-means it is the cluster's representative.
• k: the number of clusters, chosen by the user — k-means does not discover it by itself.
• Distance metric: how similarity is measured, usually Euclidean distance. The choice shapes the clusters found.
• Convergence: k-means stops when reassigning points no longer moves the centroids.
• Dendrogram: the tree diagram from hierarchical clustering showing which groups merged at each step.
• Agglomerative vs divisive: agglomerative starts with every point alone and merges the closest pairs; divisive starts with one big cluster and splits it.

Formulas

• Euclidean distance: d = √Σ(xᵢ − yᵢ)² between points x and y.
• K-means objective (SSE): minimise Σ||x − μⱼ||², the sum of squared distances from each point to its cluster centroid μⱼ.
• Centroid update: μⱼ = mean of the points assigned to cluster j.

Worked Example

Cluster (1,1), (1,2), (5,5), (5,6) with k-means, k = 2, starting centroids C1 = (1,1), C2 = (5,5).

1. Assign each point to the nearest centroid: (1,2) is distance 1 from C1 and 5 from C2 → cluster 1; (5,6) is distance 1 from C2 → cluster 2.
2. Recompute centroids: C1 = mean of (1,1),(1,2) = (1, 1.5); C2 = mean of (5,5),(5,6) = (5, 5.5).
3. Reassign: nothing moves — (1,2) is 0.5 from C1, (5,6) is 0.5 from C2. Converged.

Final clusters: {(1,1), (1,2)} centred at (1, 1.5) and {(5,5), (5,6)} centred at (5, 5.5). One assignment pass plus one centroid update sufficed, since the groups were well separated.

Common Misconceptions

• "K-means finds the true number of clusters." k is an input you must choose (elbow plot, domain knowledge). A wrong k gives confidently wrong groups.
• "K-means works for any cluster shape." It assumes roughly spherical, equally sized clusters; stretched or ring-shaped groups get chopped up.
• "Hierarchical clustering needs k upfront." It builds the full merge tree; you cut the dendrogram wherever groups look sensible.

Real-World Applications

• Marketing: customer segmentation for targeted campaigns.
• Image processing: colour quantisation compresses images by clustering similar colours.
• Anomaly detection: points fitting no cluster well are flagged as outliers, useful in fraud detection.

Clustering discovers hidden groups in unlabeled data: k-means iterates between assigning points to the nearest centroid and moving centroids to their members' mean, while hierarchical clustering builds a merge tree you cut at any level. Choose k deliberately, scale your features, and always plot the result.