Sparsity-Preserving Encodings for Straggler-Optimal Distributed Matrix Computations at the Edge

Matrix computations are a fundamental building block of the edge computing systems, with a major recent uptick in demand due to their use in AI/ML training and inference procedures. Existing approaches for distributing the matrix computations involve allocating coded combinations of submatrices to worker nodes, to build resilience to slower nodes, called stragglers. In the edge learning context, however, these approaches will compromise sparsity properties that are often present in the original matrices found at the edge server. In this study, we consider the challenge of augmenting, such approaches to preserve input sparsity when distributing the task across the edge devices, thereby retaining the associated computational efficiency enhancements. First, we find a lower bound on the weight of coding, i.e., the number of submatrices to be combined to obtain coded submatrices to provide the resilience to the maximum possible number of straggler devices (for given number of devices and their storage constraints). Next, we propose distributed matrix computation schemes which meet the exact lower bound on the weight of the coding. Numerical experiments conducted in amazon Web services (AWSs) validate our assertions regarding straggler mitigation and computation speed for the sparse matrices.