QTTARNN: A Highly‐Efficient Attention Driven Quantized Tensor Train Recursive Neural Network for Cyber‐Physical‐Social Intelligence

ABSTRACT Cyber‐Physical‐Social System (CPSS) which refers to the complex interaction of cyber, physical and social systems, has the important purpose to provide personalized intelligent services. CPSS data, generated from every aspect of people living life, are mainly in the form of time series multimodal data with characteristic of high order and high dimension. How to efficiently process these CPSS data is one of the fundamental ways for the intelligent services. In this paper, a highly‐efficient attention driven Quantized Tensor Train Recursive Neural Network is proposed, in which the CPSS data is decomposed into the form of tensor train cores. In this way, the proposed method is composed of lightweight high‐order neural network units, which better preserves the multi‐attribute features of the original data and the correlation between different dimensions by using tensors with its calculations, and implicitly trims the dense vector‐matrix connections in the fully connected network by using the form of quantization tensor train decomposition, which greatly reduces the model parameters, shortens the training time and improves the efficiency. Also, an effective attentional feature enhancement module is constructed to assist the high‐order neural network, so that the overall model can achieve a balance between low parameter number and accuracy. The network structure proposed in this paper realizes an efficient and lossless high‐order tensor recurrent neural network model with a small number of parameters. Finally, experiments on the UCF50 action video dataset, CWRU bearing dataset, and image generation tasks are conducted. Comparative analyses with vanilla LSTM and other tensorized LSTMs in terms of training time, accuracy, error, and compression ratio validate the reliability of the proposed model.