Why HDBSCAN*'s flat clustering changes with minPts: shrinking clusters, a one-run predictor and a certificate

Raise the density parameter m_pts (minPts; min_samples in the hdbscan and scikit-learn libraries) of HDBSCAN* by one small step, together with the minimum cluster size m_clSize (min_cluster_size), and a cluster of the flat partition may vanish. On benchmark data 16.9% of such steps changed the partition, and in single-cell, BERTopic and Gaia pipelines most of these changes altered the number of clusters. We explain why, show which cluster is at risk from one run, and prove when the partition cannot change. The cause is the cluster tree, not the excess-of-mass (EOM) extraction: a design that holds either fixed shows that a selected cluster shrinks below m_clSize and drops out of EOM's candidates, and planted clusters just above that threshold were lost far more often than clusters twice its size. The cluster at risk is named by counting its points whose core distance at the next value stays within its birth radius. On new tables, clusters flagged this way were lost in 82% of cases, against 10% for others of the same size. A proved sufficient condition, also checked from one run, certifies that the step leaves the partition unchanged outside an explicit tolerance set; it was never violated. Rerunning HDBSCAN* shows what changed only after a full fit per value; the predictor says beforehand which cluster is at risk and why, and the certificate proves stability, both from the run in hand plus one nearest-neighbour query. The mechanism, the cluster-level predictor and the soundness of the certificate were each confirmed on new data under plans frozen in advance. The benchmark rate of changed partitions was confirmed only narrowly (p = 0.009 against a Holm threshold of 0.010) and not in the application pipelines (13.6%, 95% CI 8.9–19.1%); the step-level screen and the comparison of the certificate with simpler rules are post hoc. The abstract in the PDF is the submitted version; this description reflects a later check. This is the manuscript as submitted to Data Mining and Knowledge Discovery on 2026-11-06 (not peer reviewed).

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-07
DOI
https://doi.org/10.5281/zenodo.24039671
Primary Topic
Advanced Clustering Algorithms Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Why HDBSCAN*'s flat clustering changes with minPts: shrinking clusters, a one-run predictor and a certificate

Felipe Provezano Coutinho
Zenodo (CERN European Organization for Nuclear Research)
Advanced Clustering Algorithms Research
preprint

Why HDBSCAN*'s flat clustering changes with minPts: shrinking clusters, a one-run predictor and a certificate

Felipe Provezano Coutinho
preprint en

Abstract

Raise the density parameter m_pts (minPts; min_samples in the hdbscan and scikit-learn libraries) of HDBSCAN* by one small step, together with the minimum cluster size m_clSize (min_cluster_size), and a cluster of the flat partition may vanish. On benchmark data 16.9% of such steps changed the partition, and in single-cell, BERTopic and Gaia pipelines most of these changes altered the number of clusters. We explain why, show which cluster is at risk from one run, and prove when the partition cannot change. The cause is the cluster tree, not the excess-of-mass (EOM) extraction: a design that holds either fixed shows that a selected cluster shrinks below m_clSize and drops out of EOM's candidates, and planted clusters just above that threshold were lost far more often than clusters twice its size. The cluster at risk is named by counting its points whose core distance at the next value stays within its birth radius. On new tables, clusters flagged this way were lost in 82% of cases, against 10% for others of the same size. A proved sufficient condition, also checked from one run, certifies that the step leaves the partition unchanged outside an explicit tolerance set; it was never violated. Rerunning HDBSCAN* shows what changed only after a full fit per value; the predictor says beforehand which cluster is at risk and why, and the certificate proves stability, both from the run in hand plus one nearest-neighbour query. The mechanism, the cluster-level predictor and the soundness of the certificate were each confirmed on new data under plans frozen in advance. The benchmark rate of changed partitions was confirmed only narrowly (p = 0.009 against a Holm threshold of 0.010) and not in the application pipelines (13.6%, 95% CI 8.9–19.1%); the step-level screen and the comparison of the certificate with simpler rules are post hoc. The abstract in the PDF is the submitted version; this description reflects a later check. This is the manuscript as submitted to Data Mining and Knowledge Discovery on 2026-11-06 (not peer reviewed).

Zenodo (CERN European Organization for Nuclear Research)
Advanced Clustering Algorithms Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Why HDBSCAN*'s flat clustering changes with minPts: shrinking clusters, a one-run predictor and a certificate — Felipe Provezano Coutinho · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS