Combining Values, Trends, and Types of Sensors for Multivariate Time-Series Classification and Regression

Although neural network-based approaches succeed in time-series classification and regression tasks, they usually ignore the trend information in the data. The primary reason is that the complex numerical trends obtained by differencing or seasonal-trend decomposition (STL) in previous studies are difficult to learn by neural networks. Moreover, to obtain the trend information in time-series sensor data, each sensor requires to be analyzed separately, which makes it challenging to retain the sensor-specific information while not increasing the number of model parameters significantly. To fill the gap above, this article first replaces complex numerical trends with concise trend states represented by trainable embedding vectors. Then, each sensor is represented by a unique trainable embedding vector and combine it with its value and trend features, so that the sensor-specific information can be preserved with only a few extra parameters. Moreover, this article also proposes masked model-based pretraining tasks suitable for multivariate time series, which solve the insufficient training problem caused by the lack of labeled data. Experiments on public datasets demonstrate that the VTSAPF model outperforms state-of-the-art methods on both time-series classification and regression tasks. The code is publicly available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ao484628/VTSAPF</uri>.