PhD Forum: Towards Metastable-Failure-Free Distributed Transaction Systems

Metastable failures in distributed databases, characterized by their self-sustaining feedback loops leading to significant performance degradation, have become increasingly prevalent with the rise of complex distributed systems [1]. One of the main sustained feedback loops in these failures is retry storms. These failures are triggered by temporary changes in load, leading to a cascade of retrial requests that overwhelm the system even after the initial load spike has recovered [2]. We have leveraged queuing theory to propose an analytical method for modeling metastable failures due to retry storms [3]. Building on our previous work, this proposal outlines a systematic approach to mitigate these failures by eliminating retrial requests in distributed transaction systems. We focus on existing concurrency control mechanisms where retries of distributed transactions can occur frequently. Specifically, we focus on two-phase locking (2PL) under high contention workloads, where many distributed transactions can abort due to deadlocks and be retried, causing metastable failures. We propose that by preprocessing distributed transactions, we can reorder the locking mechanism to avoid deadlocks and transaction retries under high contention workloads. The behavior and correctness of this approach will be validated using the queuing model we developed.