Lattice QCD code on GPUs: Implementation and performancecomparison with OpenACC and CUDA

Many recent supercomputers employ GPUs as arithmetic accelerators, and thus exploiting their performance is essential in large-scale numerical simulations.Our target application is lattice QCD simulations, in which a linear equation solver for large sparse matrix consumes most of simulation time.To implement an offloading code, there are two approaches: employing a directive-based library such as OpenACC and OpenMP, or an API-based library such as NVIDIA CUDA.In general the latter achieves higher performance because it enables detailed tuning suitable to the hardware feature.However, thanks to the development of compiler technology, the former also achieves enough performance in many practical cases.In this paper we offload the linear equation solver in lattice QCD by implementing OpenACC and CUDA codes and compare their performance on recent GPUs of the NVIDIA Hopper architecture.In particular, scaling behavior for multiple process is our interest focusing on the large-scale simulations.It would give a guide in choosing an offloading scheme for developers of similar types of application in which the memory bandwidth and communication overhead restrict the efficiency.