Memory Sandbox: A Versatile Tool for Analyzing and Optimizing HBM Performance in FPGA

Main memory access has become an increasing performance bottleneck for traditional and High-Performance Computing (HPC) applications. High Bandwidth Memory (HBM) has emerged as an alternative to conventional DRAMs, offering higher bandwidth, lower power consumption, and greater integration capabilities to meet the escalating demands of contemporary applications. The transition to HBM of the most advanced Field Programmable Gate Arrays (FPGAs) marks a paradigm shift. However, users face substantial challenges due to the scarce technical documentation and tools supporting HBM on FPGAs. This paper introduces the Memory Sandbox, an open-source tool integrated into our FPGA-Shell<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>, designed to address the complexity of utilizing HBM in FPGAs. It allows developers and students to explore roofline performances while gaining insights to improve their designs. The Memory Sandbox enables users to configure various parameters such as the number of processing elements accessing memory, the access pattern (sequential, pseudo-random, or sparse-wise), and memory configurations, emulating multiple processor threads in diverse heterogeneous scenarios. It provides detailed analyses of memory access impacts in terms of latency and throughput for scenarios with accesses within and across HBM pseudo-channels; as well as HBM performance under concurrent access scenarios. Our results show that HBM achieves 99.99% of its nominal peak bandwidth with long sequential accesses but drops to 0.17% with random data access patterns. The tool also highlights the impact of multiple AXI ports targeting the same pseudo-channel, revealing that the aggregated throughput remains constant regardless of the pseudo-channel count. Furthermore, we validate the Memory Sandbox’s capabilities by effectively profiling complex access patterns like Sparse Matrix-Vector (SpMV), demonstrating its effectiveness in providing accurate performance insights.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>https://github.com/MEEPproject/fpga_shell