PakSupercomputer: an open-source, scalable, and heterogeneous supercomputing platform

The recent advances in large vision and language models, along with generative artificial intelligence, have led to a growing demand for compute resources such as increased processing capacity, faster memory, and larger storage. In the current scenario, supercomputing is the only domain capable of solving complex and compute-intensive algorithms for large datasets. Thus, supercomputers are considered the backbone of all fields of academia, research, innovation, and commercialization. In this work, we propose and develop a Scalable Heterogeneous Supercomputer called PakSupercomputer. The PakSupercomputer uses heterogeneous processing cores, including multiple CPUs, Tensor Processing Units (TPUs), and Field Programmable Gate Arrays (FPGAs). An Elastic Parallel Programming framework is deployed to provide an easy-to-use programming environment. This programming environment assists high-performance computing developers in writing applications without dealing with the complexities of heterogeneous distributed hardware architectures. The results confirm that when testing benchmarks with larger-scale, complex problems and cloud computing applications, the PakSupercomputer proves to be a low-power, Quadrillion Floating-Point Operations Per Second (FLOPS) supercomputer.