Deep Reinforcement Learning for Automated Stock Trading Using Different Reward Schemes

In this study, we propose a deep reinforcement learning method for stock portfolio optimization. Previous work trained three actor-critic-based algorithms (PPO, A2C and DDPG) to generate portfolio weightings for the 30 stocks found in the Dow Jones Industrial Average and trained the networks to optimize for a maximization of the portfolio value. Risk consideration is not applied when training the agents; instead, it is incorporated externally during testing. Our study extends this work by training the agents on two new reward functions that incorporate risk: the Sharpe ratio and the portfolio variance. In addition, we also study the effects of using different lengths of prior data in calculating these metrics. While none of the agents trained using the risk-incorporated reward functions outperformed the agents trained using the portfolio value reward function in average cumulative return, the agents trained using the risk-based functions showed lower standard deviations in cumulative return across all experiments, suggesting the use of these reward functions to be more stable approaches to portfolio optimization. Our best results show a 9.3% average annualized return using a Sharpe ratio reward function that uses a 15-day window of historical data for calculation and a 10.4% average annualized return using a portfolio variance reward function that uses a 5-day window of historical data.