The unsupervised learning article starts from the defining condition: raw data, no labels, and an algorithm asked to find structure on its own. This workbook covers the three structures worth finding: clusters (k-means, hierarchical), a faithful low-dimensional picture (t-SNE, PCA), and topics in text (TF-IDF, NMF), ending with the recommender pipeline that ties them together. The recurring lesson arrives early and never leaves: unscaled features quietly ruin everything downstream, so the scaler is part of the model.
1. K-means: find the groups
K-means partitions data into k clusters by assigning each point to its nearest centroid and moving the centroids until they settle. With no labels involved, the model discovers grouping on geometry alone.
from sklearn.datasets import load_irisfrom sklearn.cluster import KMeansX, _ = load_iris(return_X_y=True) # ignore the labels: we are unsupervisedkmeans = KMeans(n_clusters=3, n_init=10, random_state=42)labels = kmeans.fit_predict(X) # fit AND assign in one callprint("first 10 assignments:", labels[:10])print("cluster sizes:", [list(labels).count(c) for c in range(3)])print("centroids (first 2 features):")print(kmeans.cluster_centers_[:, :2].round(2))
fit_predict learns the centroids and hands back a cluster number per row, and cluster_centers_ shows where each cluster lives in feature space. The choice of n_clusters=3 was ours, not the algorithm’s, which raises the obvious question the next example answers: how do you pick k when nobody tells you?
2. Choosing k: inertia and the elbow
Inertia measures how tightly each cluster hugs its centroid, and it always falls as k rises. The trick is finding the elbow: the k after which adding clusters stops buying much tightness.
from sklearn.datasets import load_irisfrom sklearn.cluster import KMeansX, _ = load_iris(return_X_y=True)print(" k inertia drop")previous = Nonefor k in range(1, 8): km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(X) drop = "" if previous is None else f"-{previous - km.inertia_:.0f}" print(f"{k:2d} {km.inertia_:7.0f} {drop}") previous = km.inertia_# the drops: huge, huge, then suddenly small -> the elbow sits at k=3
Read the drop column: going from 1 to 2 clusters and 2 to 3 buys enormous reductions, then the gains collapse to scraps. That bend is the elbow, and here it points at k=3, matching the three iris species we pretended not to know about. Inertia alone can never choose k, since it always prefers more clusters; the elbow is the judgement call it supports.
3. Evaluating clusters against reality
When ground truth exists, a cross-tabulation shows how the discovered clusters line up with the real categories, cluster by cluster, without the model ever having seen them.
import pandas as pdfrom sklearn.datasets import load_irisfrom sklearn.cluster import KMeansiris = load_iris()labels = KMeans(n_clusters=3, n_init=10, random_state=42).fit_predict(iris.data)crosstab = pd.crosstab( pd.Series(labels, name="cluster"), pd.Series(iris.target_names[iris.target], name="species"),)print(crosstab)# a good clustering: each row dominated by ONE species
A clean result puts one species per row with only light bleed between two of them, and that is exactly what appears: setosa lands in a cluster of its own, while versicolor and virginica overlap slightly, because they genuinely overlap in measurement space. The crosstab is the honest scorecard: it rewards structure actually found rather than labels memorised.
4. Scaling: the step that decides everything
K-means measures plain distance, so a feature with a big range dominates one with a small range. Scaling first, inside a pipeline, is the difference between clustering the data and clustering one loud column.
import pandas as pdfrom sklearn.datasets import load_winefrom sklearn.cluster import KMeansfrom sklearn.preprocessing import StandardScalerfrom sklearn.pipeline import make_pipelinewine = load_wine()# without scaling: proline (range ~1000) drowns out everything elseraw_labels = KMeans(n_clusters=3, n_init=10, random_state=42).fit_predict(wine.data)# with scaling: every feature speaks at the same volumepipe = make_pipeline(StandardScaler(), KMeans(n_clusters=3, n_init=10, random_state=42))scaled_labels = pipe.fit_predict(wine.data)for name, labels in [("RAW", raw_labels), ("SCALED", scaled_labels)]: ct = pd.crosstab(labels, wine.target_names[wine.target]) purity = ct.max(axis=1).sum() / ct.values.sum() print(f"{name:7s} cluster purity: {purity:.2%}")# RAW ~70%, SCALED ~97%: same algorithm, transformed result
The same k-means jumps from roughly 70 percent purity to near-perfect, and the only change was StandardScaler. The guide’s wider advice fits here: StandardScaler for ordinary features on different scales, Normalizer for directional data where only proportions matter, and MaxAbsScaler for sparse matrices that must stay sparse. Whichever applies, it belongs inside the pipeline.
5. Hierarchical clustering and the dendrogram
Hierarchical clustering merges the closest points step by step into a tree, and the dendrogram draws that tree, letting you see every possible clustering at once and cut at the height you choose.
import matplotlibmatplotlib.use("Agg")import matplotlib.pyplot as pltfrom scipy.cluster.hierarchy import linkage, dendrogram, fclusterfrom sklearn.datasets import load_irisfrom sklearn.preprocessing import StandardScaleriris = load_iris()X = StandardScaler().fit_transform(iris.data[::5]) # a readable subsamplemerges = linkage(X, method="ward") # ward: merges that keep clusters compactplt.figure(figsize=(10, 4))dendrogram(merges)plt.title("Iris dendrogram (ward linkage)")plt.savefig("dendrogram.png", dpi=150, bbox_inches="tight")print("saved dendrogram.png - open it and look for the long vertical gaps")labels = fcluster(merges, t=3, criterion="maxclust") # cut the tree into 3print("cluster sizes:", [list(labels).count(c) for c in (1, 2, 3)])
Open the PNG and read it bottom-up: each join is a merge, and the height of a join is how dissimilar the merged groups were. Long vertical stretches are natural places to cut, and fcluster makes the cut numeric. The method argument matters: ward favours compact equal-ish clusters, while single chains along nearest neighbours, and swapping it redraws the whole tree.
6. t-SNE: a map of high-dimensional data
t-SNE compresses many dimensions into two while fighting to keep neighbours together, producing a scatter plot where clusters you cannot see in 64 dimensions become visible islands.
import matplotlibmatplotlib.use("Agg")import matplotlib.pyplot as pltfrom sklearn.datasets import load_digitsfrom sklearn.manifold import TSNEdigits = load_digits() # 64-dimensional images of 0-9tsne = TSNE(n_components=2, learning_rate="auto", init="pca", random_state=42)coords = tsne.fit_transform(digits.data) # note: fit_transform ONLYplt.figure(figsize=(8, 6))scatter = plt.scatter(coords[:, 0], coords[:, 1], c=digits.target, cmap="tab10", s=8)plt.colorbar(scatter, label="digit")plt.title("t-SNE map of the digits dataset")plt.savefig("tsne_digits.png", dpi=150, bbox_inches="tight")print("saved tsne_digits.png - ten islands, one per digit")
The saved map shows the 64-dimensional digits resolving into ten coloured islands, structure that was always there and simply invisible. Two cautions from the guide: t-SNE has fit_transform but no separate transform, so it cannot map new points onto an existing plot, and its axes have no meaning, since only the grouping is trustworthy. It is a visualisation tool, not a preprocessing step.
7. PCA: how many dimensions actually matter
PCA finds the directions of greatest variance and ranks them. The explained variance per component reveals the data’s intrinsic dimension: how many directions carry real information rather than noise.
from sklearn.datasets import load_winefrom sklearn.preprocessing import StandardScalerfrom sklearn.decomposition import PCAX = StandardScaler().fit_transform(load_wine().data) # 13 featurespca = PCA().fit(X)print("component variance share cumulative")cumulative = 0for i, share in enumerate(pca.explained_variance_ratio_[:8]): cumulative += share print(f" {i} {share:.3f} {cumulative:.3f}")# the first 3 components carry ~67%; by 8 you have ~90%reduced = PCA(n_components=3).fit_transform(X)print("shape after reduction:", reduced.shape) # (178, 3) from (178, 13)
Thirteen features collapse to three components holding two-thirds of the variance, which says the wine data mostly lives on a three-dimensional sheet folded into thirteen-dimensional space. That is the intrinsic dimension, and it is the principled answer to “how many components should I keep”: read the cumulative column and stop where the gains flatten.
8. Inside PCA: components and decorrelation
Each principal component is a recipe over the original features, and its loadings tell you what the component means. PCA also decorrelates: the transformed features have zero correlation with each other.
import numpy as npfrom sklearn.datasets import load_irisfrom sklearn.preprocessing import StandardScalerfrom sklearn.decomposition import PCAiris = load_iris()X = StandardScaler().fit_transform(iris.data)pca = PCA(n_components=2).fit(X)transformed = pca.transform(X)print("PC1 loadings:")for name, weight in zip(iris.feature_names, pca.components_[0]): print(f" {name:18s} {weight:+.2f}")# petal length and width dominate: PC1 is roughly "overall petal size"corr_before = np.corrcoef(X[:, 2], X[:, 3])[0, 1]corr_after = np.corrcoef(transformed[:, 0], transformed[:, 1])[0, 1]print(f"\ncorrelation of two raw features: {corr_before:+.2f}")print(f"correlation of the two components: {corr_after:+.2f}")
The loadings read like an interpretation: PC1 weights the petal measurements heavily and in the same direction, so it is effectively a petal-size axis. And the correlation check shows the second gift: raw petal length and width correlate at +0.96, while the two components correlate at exactly zero, which is why PCA is a standard pre-step for models that dislike correlated inputs.
9. Clustering text: TF-IDF and TruncatedSVD
Text becomes clusterable through TF-IDF, which produces a sparse matrix, and sparse matrices need TruncatedSVD rather than PCA for reduction, because SVD preserves sparsity. Then k-means clusters the compressed documents.
from sklearn.feature_extraction.text import TfidfVectorizerfrom sklearn.decomposition import TruncatedSVDfrom sklearn.cluster import KMeansfrom sklearn.pipeline import make_pipelinedocuments = [ "the striker scored twice and the keeper saved a penalty", "midfield pressing won the match in the second half", "the defender was sent off before the final whistle", "quarterly revenue rose as new customers signed contracts", "the merger boosted profits and share price this quarter", "investors welcomed the earnings report and dividend rise",]tfidf = TfidfVectorizer()matrix = tfidf.fit_transform(documents)print("TF-IDF matrix:", matrix.shape, "- sparse:", type(matrix).__name__)pipe = make_pipeline( TruncatedSVD(n_components=3, random_state=42), # PCA's sparse-friendly cousin KMeans(n_clusters=2, n_init=10, random_state=42),)labels = pipe.fit_predict(matrix)for label, doc in zip(labels, documents): print(f"cluster {label}: {doc[:48]}...")# football in one cluster, finance in the other, no labels supplied
Six documents separate cleanly into football and finance without a single label, because TF-IDF gave weight to the distinctive words and SVD compressed thousands of word-columns into three dense features k-means can handle. The substitution rule to remember: dense numeric data takes PCA, sparse text matrices take TruncatedSVD, same idea, different plumbing.
10. NMF topics and a recommender
The finale: NMF factorises the document-word matrix into additive topics, each readable as its top words, and normalised NMF features power a similarity recommender via dot products, the guide’s full MaxAbsScaler-NMF-Normalizer pipeline in miniature.
import numpy as npimport pandas as pdfrom sklearn.feature_extraction.text import TfidfVectorizerfrom sklearn.decomposition import NMFfrom sklearn.preprocessing import Normalizerdocuments = [ "the striker scored and the keeper saved a late penalty", "midfield pressing and a corner won the football match", "the defender scored from a corner in the final minute", "quarterly revenue rose as new customers signed contracts", "the merger boosted profits and the share price rose", "investors welcomed the earnings report and dividend growth",]tfidf = TfidfVectorizer()matrix = tfidf.fit_transform(documents)words = tfidf.get_feature_names_out()nmf = NMF(n_components=2, random_state=42)doc_topics = nmf.fit_transform(matrix) # documents x topics# read each topic through its heaviest wordsfor t, row in enumerate(nmf.components_): top = [words[i] for i in np.argsort(row)[-4:][::-1]] print(f"topic {t}: {', '.join(top)}")# topic 0: football words; topic 1: finance words# recommender: normalise, then dot products = cosine similaritynormed = Normalizer().fit_transform(doc_topics)similarities = pd.Series(normed @ normed[0], index=range(len(documents)))print("\nmost similar to document 0:")print(similarities.sort_values(ascending=False).head(3))# documents 1 and 2 (the other football texts) top the list
NMF’s non-negativity is what makes the topics readable: every document is built by adding topics, never subtracting, so the components decompose into parts, sports words in one, finance words in the other. The recommender is three lines because the geometry does the work: normalise the topic vectors to length one and a dot product becomes cosine similarity, so “most similar document” is just the largest number. This same pipeline, scaled up, is how article and product recommenders actually start life.
Work through these and you have the whole article in practice: k-means with the elbow and crosstab, scaling as the make-or-break step, hierarchical trees and where to cut them, t-SNE maps, PCA’s variance, loadings, and decorrelation, sparse text through TruncatedSVD, and NMF topics feeding a cosine recommender. The guide’s closing decision tree compresses to three questions worth memorising: want groups, cluster; want a picture, t-SNE; want fewer or more meaningful dimensions, PCA for dense data, TruncatedSVD for sparse, and NMF when the parts need to be readable.
See you soon, Andrei.
[…] Unsupervised Machine Learning: 10 Code-Along Examples […]
[…] Unsupervised Machine Learning: 10 Code-Along Examples […]
[…] Code along: https://datalad.co.uk/unsupervised-machine-learning-10-code-along-examples/ […]