A Reinforcement Learning Framework for Portfolio Optimization via Sequence Prediction

Portfolio optimization (PO) is to allocate capital continuously on a group of financial assets, in order to trade off between risk and returns to achieve investors' satisfactory return on investment. A PO problem is formalized as a Markov decision process (MDP) of sequential decision and PO problem, where reinforcement learning (RL) has made the most significant progress in recent years. In this paper, we propose a novel deep reinforcement learning (DRL) framework, which is termed SFFARIMA reinforcement learning (SFFRL) aided by SFF (sine functions fitting), ARIMA (auto-regressive integrated moving average) and proximal policy optimization (PPO), to tackle the PO problem. SFFRL integrates DRL with time sequence prediction, which takes the predicted time sequence as the augmented part of the input state of the policy network to achieve a better training effect. SFFRL combines the advantages of sine functions fitting (SFF) and auto-regressive integrated moving average (ARIMA) to achieve better forecasting prediction. Note that SFF is sensitive to stock price volatility, and ARIMA could predict the stock price trend well. By conducting experiments on stock datasets from S&P 500 (i.e., Standard & Poor's 500) constituent stocks, we backtest that SFFRL in 2013-2025. SSFRL outperforms all the other benchmark models or algorithms regarding risk control and investment returns in the all-time, long-term, medium-term, and short-term.