Skip to main content

Common mistakes and best practices in agglomerative clustering

Agglomerative clustering creates a merge hierarchy under a chosen representation, metric, and linkage. A dendrogram is a faithful picture of that procedure; it is not proof that the displayed taxonomy is uniquely real.

1. Ignoring feature scale and meaning​

Distance-based merging is sensitive to units. Standardization is appropriate only when unit variance represents a sensible relative weight. Categorical codes, mixed data types, and domain-weighted measurements may need a different dissimilarity instead.

import numpy as np
from sklearn.cluster import AgglomerativeClustering
from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler

X, reference_groups = make_blobs(n_samples=150, centers=3, cluster_std=0.60, random_state=42)
X[:, 1] *= 100
X_scaled = StandardScaler().fit_transform(X)
ward = AgglomerativeClustering(n_clusters=3, metric="euclidean", linkage="ward")
labels_ward = ward.fit_predict(X_scaled)
assert np.unique(labels_ward).size == 3

In a deployed analysis, fit preprocessing on the reference/training population and reuse it. Otherwise the same observation can move in the geometry merely because a new evaluation batch arrived.

2. Mixing incompatible metrics and linkages​

Single uses the nearest cross-cluster pair and can chain through bridges. Complete uses the farthest pair and favors compact groups but is sensitive to extremes. Average uses the mean cross-cluster distance. Ward greedily minimizes the increase in within-cluster variance and accepts only Euclidean geometry in scikit-learn.

partitions = {}
for linkage_name in ("single", "complete", "average", "ward"):
    model = AgglomerativeClustering(
        n_clusters=3,
        metric="euclidean",
        linkage=linkage_name,
    )
    partitions[linkage_name] = model.fit_predict(X_scaled)
assert all(np.unique(labels).size == 3 for labels in partitions.values())

Do not choose the linkage that merely gives the prettiest plot. Define the cluster notion first, then assess robustness and downstream usefulness.

3. Reading a dendrogram without its scale​

SciPy's linkage matrix records child nodes, merge distance, and merged observation count. Long vertical gaps can suggest candidate cuts, but there is no automatic universally optimal cut.

import matplotlib.pyplot as plt
from scipy.cluster.hierarchy import dendrogram, linkage

linked = linkage(X_scaled, method="ward", metric="euclidean")
assert linked.shape == (149, 4)

fig, ax = plt.subplots(figsize=(9, 4))
dendrogram(linked, truncate_mode="lastp", p=12, show_leaf_counts=True, ax=ax)
ax.set(title="Truncated Ward dendrogram", xlabel="merged node or observation", ylabel="Ward distance")
fig.tight_layout()
plt.close(fig)

SciPy and scikit-learn perform separate fits here. Keep their metric and linkage settings aligned before comparing a tree with flat labels.

4. Treating one cluster count as discovered truth​

Compare plausible cuts using domain meaning, stability, and internal diagnostics. Silhouette evaluates cohesion and separation under the chosen metric and tends to favor convex separation; it cannot validate the hierarchy itself.

from sklearn.metrics import silhouette_score

scores = {}
for n_clusters in (2, 3, 4, 5):
    labels = AgglomerativeClustering(
        n_clusters=n_clusters, metric="euclidean", linkage="ward"
    ).fit_predict(X_scaled)
    scores[n_clusters] = silhouette_score(X_scaled, labels)
assert max(scores, key=scores.get) == 3

The known optimum is specific to this separated synthetic fixture. Real analyses should test whether nearby cuts tell a stable and useful story.

5. Ignoring noise, ties, and irreversibility​

Agglomerative merges are greedy and cannot be undone. Outliers can cause late, high-distance merges; single linkage can chain through sparse bridges. Exact distance ties can admit multiple equally valid trees whose leaf order or merge order differs. Inspect sensitivity to perturbations rather than overinterpreting one branch order.

6. Underestimating resource use​

General hierarchical clustering needs substantial pairwise-distance and tree storage and is usually less suitable than K-Means for very large sample counts. A truncated dendrogram only changes rendering; it does not make building the full linkage matrix cheap. Connectivity constraints or early stopping can help in structured cases, but they also change the fitted problem.

7. Use label-invariant external checks when available​

from sklearn.metrics import adjusted_rand_score

ari = adjusted_rand_score(reference_groups, labels_ward)
assert ari > 0.95

Adjusted Rand index ignores arbitrary label numbering, but reference labels must be independently meaningful. When none exist, report the analysis as exploratory.

Conclusion​

State preprocessing, metric, linkage, stopping/cut rule, resource limits, and validation evidence. Hierarchical output is most useful when the merge structure itself matters and remains reasonably stable under defensible perturbations.