3D long time spatiotemporal convolution for complex transfer sequence prediction

Spatiotemporal sequences prediction(SSP) aims to predict the future situation in a period of time based on the spatiotemporal sequences data(SSD) of historical observations. In recent years, deep learning-based models have received more attention and research in SSP tasks. However, two challenges still exist in the existing methods: 1) Most of the existing spatio-temporal prediction tasks focus on extracting temporal information using recurrent neural networks and using convolution networks to extract spatial information, but ignore the fact that the forgetting of historical information still exists as the input sequence length increases. 2) Spatio-temporal sequence data have complex non-smoothness in both temporal and spatial, such transient changes are difficult to be captured by existing models, while such changes are often particularly important for the detail reconstruction in the image prediction task. In order to solve the above problems, we propose 3DcT-Pred based on the existing mainstream end-to-end modeling framework and combined with two-branch 3D convolution.Specifically, the proposed model first mitigates the long-range forgetting problem by extracting long-term global features of spatio-temporal sequence data. Secondly, a cross-structured spatio-temporal attention module is constructed based on spatio-temporal features in the decoding stage to enhance the response of fine features in the image in the convolutional channel, so as to capture non-smooth local features. Finally, a fusion gating module is designed to integrate global and local features, thereby improving the model performance. We conducted comparative experiments with state-of-the-art models on three publicly available datasets and a private radar echo dataset. Experimental results show that the proposed model achieves good results on the SSP tasks.