Characterising CPU-initiated High Bandwidth Memory Access

Abstract An increasing number of high-performance computers are equipped with high bandwidth memory. Their CPUs have cache-coherent access to this memory type over high-speed interconnects or as unified memory. We analyse two such systems, NVIDIA’s GH200 and AMD’s MI300A. We discuss their memory setup and measure memory access latency and throughput. We evaluate weighted memory page interleaving and strided memory access as candidates for increasing the measured throughput. Weighted interleaving utilises CPU and GPU memory simultaneously and enables exceeding the measured throughput on only CPU or GPU memory for all tested workloads. Using strided memory access, which accesses memory at fixed intervals, we achieve higher throughput than using a non-strided sequential pattern.