Exploring high-level patterns for portfolio selection through pre-training

Transformer-based policy networks have shown promising prospects in Portfolio Selection (PS). However, constrained by high computational complexity, they primarily focus on short-term price series (e.g., the past 12 h), while overlooking valuable long-term features. Furthermore, the inherent volatility and noise tend to lead the model to overfit on low-level information. To tackle these challenges, we propose an innovative approach. This approach leverages a pre-training model to extract high-level patterns from extended price series (e.g., the past two weeks) and then enhances decision-making in policy networks. The pre-training model is designed based on discrete representations to achieve better generalization and interpretability. Specifically, we tokenize the price series into discrete tokens through Vector-Quantization Variational AutoEncoder (VQ-VAE) and encourage the pre-training model to reconstruct these tokens according to the masked price series. Then we introduce a compact encoder-only policy network named Portfolio Transformer Encoder (PTE). Finally, PTE is provided with high-level patterns from the pre-training model to make more comprehensive decisions. We term the whole approach as Vector-Quantization Portfolio Transformer Encoder (VQ-PTE). VQ-PTE demonstrates superior performance on real-world currency and S&P500 datasets, achieving a minimum improvement of 35% in returns. Additionally, visualization results highlight superior interpretability.