Scaling molecular dynamics for large-scale simulation of biological systems on AMD CPU/GPU supercomputers: Lessons from LUMI

This paper presents optimizations of the GENESIS molecular dynamics (MD) software for LUMI. GENESIS is a state-of-the-art MD program capable of simulating large-scale biological systems, yet it requires significant optimizations and investigations to fully utilize the computational resources of the LUMI-G hardware partition of LUMI, a hybrid architecture featuring AMD EPYC processors and AMD Instinct GPUs. Our work focuses on (1) addressing CPU-GPU communication bottlenecks, (2) tuning electrostatic interaction calculations using the Particle Mesh Ewald (PME) method, and (3) adapting GENESIS kernels for the AMD GPUs. A significant improvement in the MD performance is achieved through the NUMA mapping on each LUMI-G node. This improvement is shown in benchmark results for different molecular systems comprising from 2×104 to 1.6×107 atoms. The reciprocal-space calculation in the PME method on LUMI-G is faster than that on FUGAKU. Since the reciprocal-space calculation is carried out using only CPUs on both LUMI-G and FUGAKU, the results highlight the superior network performances on the former. The overall performance of a large system containing 1.6×107 atoms shows 200.8 ns/days when using 1024 nodes on LUMI-G, which is a better performance than previously reported results using general-purpose supercomputers. We conclude by providing recommendations for the optimal configurations of GENESIS on LUMI-G, including guidelines for selecting PME schemes for different system sizes and computational requirements. This optimization allows us to study cellular systems' stability, dynamics, and function relationships using atomistic MD simulations on a large scale.