Deep Latent Variable Predictive Modeling With Online Bayesian Soft Attention Mechanism

Inspired by the idea of deep learning, several latent variable models have been successfully extended to the deep forms for industrial data analytics. Compared to traditional deep neural networks, deep latent variable models are more fitted to the requirement of data-driven modeling and applications in industrial production systems, due to the lightweight model structure and high-efficient data analytics process. The aim of this article is to develop a deep latent variable predictive modeling framework, which is based on a newly designed Bayesian soft attention mechanism. Instead of only using extracted features from the last hidden layer for predictive modeling, different attentions are focused across all hidden layers of the deep model. As a result, different layer-wise models make different contributions in predicting different data patterns, through which the ability of the deep latent variable model will be further explored. Two real industrial examples are provided for performance evaluation and comparative studies among different prediction models, based on which the superiority of the online Bayesian soft attention mechanism has been confirmed.