Why HDBSCAN*'s flat clustering changes with minPts: shrinking clusters, a one-run predictor and a certificate
Raise the density parameter m_pts (minPts; min_samples in the hdbscan and scikit-learn libraries) of HDBSCAN* by one small step, together with the minimum cluster size m_clSize (min_cluster_size), and a cluster of the flat partition may vanish. On benchmark data 16.9% of such steps changed the partition, and in single-cell, BERTopic and Gaia pipelines most of these changes altered the number of clusters. We explain why, show which cluster is at risk from one run, and prove when the partition cannot change. The cause is the cluster tree, not the excess-of-mass (EOM) extraction: a design that holds either fixed shows that a selected cluster shrinks below m_clSize and drops out of EOM's candidates, and planted clusters just above that threshold were lost far more often than clusters twice its size. The cluster at risk is named by counting its points whose core distance at the next value stays within its birth radius. On new tables, clusters flagged this way were lost in 82% of cases, against 10% for others of the same size. A proved sufficient condition, also checked from one run, certifies that the step leaves the partition unchanged outside an explicit tolerance set; it was never violated. Rerunning HDBSCAN* shows what changed only after a full fit per value; the predictor says beforehand which cluster is at risk and why, and the certificate proves stability, both from the run in hand plus one nearest-neighbour query. The mechanism, the cluster-level predictor and the soundness of the certificate were each confirmed on new data under plans frozen in advance. The benchmark rate of changed partitions was confirmed only narrowly (p = 0.009 against a Holm threshold of 0.010) and not in the application pipelines (13.6%, 95% CI 8.9–19.1%); the step-level screen and the comparison of the certificate with simpler rules are post hoc. The abstract in the PDF is the submitted version; this description reflects a later check. This is the manuscript as submitted to Data Mining and Knowledge Discovery on 2026-11-06 (not peer reviewed).
Authors
- Felipe Provezano Coutinho (ORCID: https://orcid.org/0000-0001-7206-5856)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-07
- DOI
- https://doi.org/10.5281/zenodo.24039671
- Primary Topic
- Advanced Clustering Algorithms Research
- Type
- preprint