Incremental clustering based on Wasserstein distance between histogram models

In this article, we present an innovative clustering framework designed for large datasets and real-time data streams which uses a sliding window and histogram model to address the challenge of memory congestion while reducing computational complexity and improving cluster quality for both static and dynamic clustering. The framework provides a simple way to characterize the probability distribution of cluster distributions through histogram models, regardless of their distribution type. This advantage allows for efficient use with various conventional clustering algorithms. To facilitate effective clustering across windows, we use a statistical measure that allows the comparison and merging of different clusters based on the calculation of the Wasserstein distance between histograms. • Fast, flexible framework for static/dynamic clustering over sliding windows. • Uses histogram models to represent clusters with arbitrary distributions. • High degree of flexibility in selecting the clustering algorithm to apply. • Achieves efficient, high quality results with Wasserstein-based statistical tests.