Marl-Enhanced Federated Learning for Predicting Remaining Useful Lifetime

Remaining useful life (RUL) prediction is an important problem in the Prognostics and Health Management (PHM) of industrial machines. Recently, the proliferation of Internet of Things (IoT) has resulted in the availability of a large amount of sensor data obtained from the monitoring of industrial machines. This has greatly enhanced the performance of deep learning based RUL prediction methods. RUL prediction is a multivariate time series forecasting problem where state-of-the-art results have been achieved using transformer architectures and its variants. This success is based on their long sequence modelling ability. Two important characteristics of transformers is that they have a large number of free parameters and require a lot of data for training, and any single organization may not have access to the required data volume. In addition, operational sensor data is private, which organizations may not be willing to share. The Federated Learning (FL) is a suitable method to tackle this issue as it enables model training using data distributed over several sources without explicit data sharing. In many situations, FL client devices connect to the FL coordinator over wireless network and hence the performance achieved will be affected by wireless loss. Furthermore, FL clients may want to use UDP over TCP to exchange the large number of transformer model parameters. In this work, we first propose a novel transformer architecture for RUL prediction. Secondly, we propose multi-agent reinforcement learning (MARL) methods to mitigate the effect of wireless loss on FL based transformer model training. Our system formulation and experiments comprehensively demonstrate the effectiveness of our approach.