A Plug-and-Play Module for Enhancing Fault-Tolerant Distributed Inference Based on Gaussian Dropout

Deploying deep neural network (DNN) models to the Internet of Things (IoT) for distributed inference (DI) is becoming an increasingly common requirement for AI applications. When IoT transmissions occur over lossy networks, data loss can significantly affect the accuracy of inference tasks. Therefore, we propose a plug-and-play module called Loss-Adapter, which can improve the accuracy of DI on lossy networks. To simulate network packet loss, we design a Gaussian distribution sampling dropout. This idea is utilized during the training of Loss-Adapter to acquire fault-tolerant inference capability on unreliable communication links. Moreover, we use the Tree-structured Parzen Estimator algorithm to optimize the parameters of the Gaussian dropout layer. To avoiding catastrophic forgetting, we implement freeze training. In addition, as a plug-and-play module, it is very easy to train and use. Our experiments show that by applying the Loss-Adapter, the inference accuracy of DNN models on lossy IoT is significantly improved.