Fault Tolerance in Stochastic Circuits for Recurrent Sequential Neural Networks

Stochastic computing (SC) provides low-area, power-efficient hardware solutions suitable for edge systems, but its scalability poses challenges due to its precision limitations, especially in noisy environments. This paper investigates the fault tolerance of key SC components, such as stochastic number generators (SNGs) and activation functions (AFs), within recurrent and sequential networks like long short-term memory (LSTM) networks. We inject bit-flip faults into a network’s most sensitive inputs, weights and AFs, and study fault propagation across different network layers. Our findings reveal that SC component choices significantly influence fault tolerance. For example, networks using Sobol-based SNGs with tanh AFs exhibited stronger resilience than those with LFSR-based SNGs and ReLU AFs, by maintaining higher accuracy under fault conditions. However, the accuracy of the networks declined significantly under simultaneous faults in inputs, weights and AFs. Additionally, while increasing stochastic number lengths improved fault tolerance, they also increased latency. Nevertheless, the SC networks achieved up to 54% area and 72% power savings, making them ideal for resource-constrained applications. We also found that there is a trade-off between efficiency and fault tolerance: SC designs that focus on resource efficiency may struggle in noisy environments, where fault-tolerant SC designs are more effective. This implies that fault resilience in SC architectures depend heavily on design choices and cannot be assumed inherent across all configurations.