Effect of Pipelining on Resource-Aware Application-Level Checkpointing System for Malleable MPI Applications

Malleable MPI applications, which can dynamically adjust their resources during runtime, require efficient and flexible checkpointing systems to maintain fault tolerance. Traditional checkpointing methods often struggle to adapt to these changes, leading to performance bottlenecks. This paper presents iCheck, a resource-aware, application-level checkpointing system designed to support malleable applications. iCheck introduces dynamic pipelining to optimize checkpoint transfers by breaking data into smaller chunks, allowing efficient and scalable transfers. Combined with Remote Direct Memory Access (RDMA), iCheck reduces data transfer latency and minimizes memory overhead. The system adapts dynamically to resource changes by adjusting pipeline size and agent distribution, ensuring minimal disruption to application performance. The impact of pipelining on checkpointing performance is evaluated, demonstrating its ability to efficiently handle dynamic resource allocation in HPC environments. We also examine the system's behavior during component failures, where iCheck temporarily bypasses RDMA and writes checkpoints directly to the Parallel File System (PFS).