UFHDA: A Unified Framework for Heterogeneous Big Data Acceleration

Although the Spark platform has effectively alleviated the input/output (I/O) bottleneck issues present in Hadoop, the current computational performance has become a new limiting factor as application demands continue to grow. With the emergence of Graphics Processing Unit (GPU) accelerated computing technology, new avenues have opened up for addressing the computational performance bottlenecks of the Spark platform. In recent years, an increasing number of researchers have been dedicated to exploring how to leverage general-purpose acceleration hardware such as GPUs to enhance the parallel processing capabilities of big data. However, current GPU-accelerated solutions for big data primarily rely on NVIDIA GPUs, lacking broad compatibility with other types of GPUs or accelerators, and their dependence on storage systems is also becoming increasingly pronounced. To address these known issues, we have developed a unified big data adaptation framework for heterogeneous GPUs to achieve unified acceleration across different GPU types(UFHDA). Extensive testing and validation have demonstrated that this solution achieves a significant performance improvement of 5–16 times compared to traditional CPUs.