Performance Analysis of Inline Compression in pySDC

The volume of data required for High Performance Computing (HPC) jobs is growing faster than the memory storage available to store the required data, leading to performance bottlenecks. Hence the need for inline data compression, which reduces the amount of allocated memory needed by storing all data in its compressed format and decompressing/recompressing single variables as needed. We apply inline compression to the HPC application pySDC, a framework for parallel-in-time integration of partial differential equations. We introduce a new version of pySDC that has a compression manager to add inline compression functionality, along with a software cache that stores the decompressed state of the most frequently used variables. We use the ZFP lossy compressor to test our model with varying software cache sizes. Results show that having no cache has the best compression ratio (CR) at size 5.8, but having a cache size of 16 reduces total execution time by 2.6× while also slightly improving the memory footprint with a CR of 1.5. Our framework overall provides user versatility in the trade-off between execution time and memory savings.