MGPU-FI: A Fault Injection Framework for Reliability Analysis of Multi-GPU System

Graphics Processing Units (GPUs) have become indispensable in high-performance and general-purpose computing. As GPU systems are increasingly deployed in safety-critical and data-intensive domains, ensuring their reliability in the presence of soft errors has become a critical challenge. This challenge is further exacerbated in multi-GPU systems, where architectural complexity significantly increases the difficulty of tracking and mitigating error propagation across processing units. In this paper, we present a systematic study of reliability analysis in multi-GPU architectures. To achieve this goal, we develop a new fault injection framework, MGPU-FI, that simulates soft errors during application execution with fine-grained control. The framework enables comprehensive tracking of injection events, memory access patterns, and propagation traces across GPUs. We also introduce a visualization model for cross-GPU error propagation, enabling future systematic analysis of error propagation patterns in multi-GPU environments. We conduct comprehensive fault injection experiments using MGPU-FI across 14 multi-GPU workloads. Our analysis systematically evaluates soft error resilience across cross-GPU propagation patterns, bit location, and data type, establishing new insights into resilience bottlenecks for multi-GPU architectures.