Decomposing masked time series model: a self-supervised time-series forecasting framework

Today, time series forecasting is widely used in a variety of areas. Self-supervised pre-training has drawn great attention, which have obviously improved the accuracy in time series forecasting tasks and have reduced labeling costs. Motivated by methods in NLP or CV, a self-supervised paradigm is random masking & reconstruction, which guides representation learning by using non-masked tokens to reconstruct the masked ones. Contrastive learning is also an effective selfsupervised learning method for time series, forcing models to learn discriminative representations from different views of input time series. Time series decomposition is frequently applied to time series related tasks to obtain disentangled seasonal and trend parts, benefiting network’s abilities of capturing periodic patterns and long-term series modeling. Therefore, we design DMTM, a self-supervised pre-training framework based on Transformer encoder. It obtains highfrequency and low-frequency part of the original time series via FFT decomposition firstly. Then different pre-training tasks are proposed for high-frequency and low-frequency part according to noticeably different features of the two decomposed parts. Extensive experiments conducted on multiple real-world public datasets have shown that our model outperforms the current state-of-the-art methods in time series forecasting tasks, improving performance of forecasting by 9.0% on average when using unified backbone encoder.