Computational Offloading and Caching Framework for Parallel and Distributed Edge Computing in Supercomputing Environments

Despite significant advancements in parallel and vector supercomputers over the past twenty years, the present generation falls short of expected performance benchmarks due to programming and scaling issues. This paper examines improvements in parallel processing, emphasizing how hierarchical parallelism and improved clustering algorithms might enhance the performance of contemporary supercomputers to meet user expectations. This study investigates the iHPI-KC algorithm, an enhanced variant of the K-means clustering technique that incorporates hierarchical parallelism to optimize computational offloading and cache scalability on many-core supercomputers, particularly the Sunway TaihuLight. The iHPI-KC approach integrates a novel hyper-parameter selection procedure (HPSP), providing three unique layers of parallelism: dataflow partitioning, centroid partitioning, and a combined partitioning across dataflow, centroids, and dimensions. Evaluation of datasets from the UCI repository and Kaggle demonstrates that the proposed system attains enhanced caching scalability (96.53%), computational offloading efficacy (94.53%), and adaptability (97.53%) relative to traditional models, simultaneously decreasing processing duration and improving execution velocity. This study emphasizes the prospects of hierarchical parallelism and algorithmic improvements to enhance supercomputing efficiency.