Design and Implementation of Cluster Analysis System under Big Data Platform

Based on the distributed computing architecture, this research combines Apache Spark and Flink to optimise batch processing and streaming computation, and adopts algorithms such as K-Means, DBSCAN, and hierarchical clustering to perform efficient clustering analysis for different data characteristics. GPU acceleration, cache optimisation and indexing techniques are introduced to improve the computation performance, and a visual analysis interface is provided to enhance the user interaction experience. In this paper, we will elaborate the system architecture design, key technology implementation, experimental evaluation and optimisation strategy to explore the feasibility and superiority of the cluster analysis system in the big data environment.