Large data density peak clustering based on sparse auto-encoder and data space meshing via evidence probability distribution

The development of big data analysis technology has brought new development opportunities to the production and management of various industries. Through the mining and analysis of various data in the operation process of enterprises by big data technology, the internal associated data of the enterprises and even the entire industry can be obtained. As a common method for large-scale data statistical analysis, clustering technology can effectively mine the relationship within massive heterogeneous multidimensional data, complete unlabeled data classification, and provide data support for various model analysis of big data. Common big data density clustering methods are time-consuming and easy to cause errors in data density allocation, which affects the accuracy of data clustering. Therefore we propose a novel large data density peak clustering based on sparse auto-encoder and data space meshing via evidence probability distribution. Firstly, the sparse auto-encoder in deep learning is used to achieve feature extraction and dimensionality reduction for input high-dimensional data matrix through training. Secondly, the data space is meshed to reduce the calculation of the distance between the sample data points. When calculating the local density, not only the density value of the grid itself, but also the density value of the nearest neighbors are considered, which reduces the influence of the subjective selection truncation distance on the clustering results and improves the clustering accuracy. The grid density threshold is set to ensure the stability of the clustering results. Using the K-nearest neighbor information of the sample points, the transfer probability distribution strategy and evidence probability distribution strategy are proposed to optimize the distribution of the remaining sample points, so as to avoid the joint error of distribution. The experimental results show that the proposed algorithm has higher clustering accuracy and better clustering performance than other advanced clustering algorithms on artificial and real data sets.