Clustering & PCA: learning without labels

Unsupervised learning finds structure with no answer key. Two tools cover most of the practical need: clustering (group similar points) and dimensionality reduction (compress and visualize).

Clustering — discover groups

• k-means: pick k, assign points to the nearest centroid, move centroids to the mean, repeat. Fast and ubiquitous. Caveats: you must choose k (use the elbow/silhouette method), it assumes round, similar-size clusters, and it's sensitive to scaling and initialization (use k-means++).
• Hierarchical / DBSCAN: build a tree of clusters, or find dense regions and label sparse points as noise (DBSCAN finds k itself and arbitrary shapes).

Uses: customer segmentation, deduplication, exploratory analysis, grouping embeddings to see what a model "thinks" is similar.

PCA — compress along the directions that matter

Principal Component Analysis finds the orthogonal directions of maximum variance and re-expresses the data in those coordinates. Keep the top few components and you shrink dimensions while preserving most of the signal.

• Fights the curse of dimensionality and decorrelates features.
• Speeds up downstream models and enables 2-D/3-D visualization.
• It's linear — it can't unfold curved structure.

PCA vs UMAP/t-SNE for visualization

Use PCA to reduce and decorrelate; use UMAP/t-SNE to look — and never read exact distances off a t-SNE plot.

Pitfall

Always scale features before k-means and PCA — both are distance/variance based, so an unscaled large-range feature dominates everything.

Connects to: dimensionality · unsupervised learning · learned representations