Incremental Reinforcement Learning for Portfolio Optimisation

Portfolio optimisation is a crucial decision-making task. Traditionally static, this problem is more realistically addressed as dynamic, reflecting frequent trading within financial markets. The dynamic nature of the portfolio optimisation problem makes it susceptible to rapid market changes or financial contagions, which may cause drifts in historical data. While reinforcement learning (RL) offers a framework that allows for the formulation of portfolio optimisation as a dynamic problem, existing RL approaches lack the ability to adapt to rapid market changes, such as pandemics, and fail to capture the resulting concept drift. This study introduces a recurrent proximal policy optimisation (PPO) algorithm, leveraging recurrent neural networks (RNNs), specifically the long short-term memory network (LSTM) for pattern recognition. Initial results conclusively demonstrate the recurrent PPO’s efficacy in generating quality portfolios. However, its performance declined during the COVID-19 pandemic, highlighting susceptibility to rapid market changes. To address this, an incremental recurrent PPO is developed, leveraging incremental learning to adapt to concept drift triggered by the pandemic. This enhanced algorithm not only learns from ongoing market data but also consistently identifies optimal portfolios despite significant market volatility, offering a robust tool for real-time financial decision-making.